Send data from Segment to BigQuery

Updated on 2022-03-14

In this article we will show you how to store your Segment data in BiqQuery. To do this, you need to follow a few steps:

  1. Enable BigQuery for your Google Cloud project
  2. Create a GCP service account for Segment to connect with
  3. Create a warehouse in the Segment app

Create a Project and Enable BigQuery

  1. Go to Google Cloud Platform
  • If you don’t have a project already, create one
  • If you have an existing project, you will need to enable the BigQuery API. Once you’ve done so, you should see BigQuery in the “Resources” section of Cloud Platform
note:

Make sure billing is enabled on your project, or Segment will not be able to write into the cluster.

  1. Copy the project ID. You will need it when you create a warehouse source in the Segment app.

Create a Service Account for Segment

Refer to Google Cloud’s documentation about service accounts for more information.

  1. From the Navigation panel on the left, go to IAM & admin > Service accounts

  2. Click Create Service Account along the top

  3. Enter a name for the service account (for example: “segment-warehouses”) and click Create

  4. When assigning permissions, make sure to grant the following roles:

  • BigQuery Data Owner
  • BigQuery Job User

  1. Create a JSON key. The contents of the downloaded file will be used to create your warehouse in the next section.

Create the Warehouse in Segment

  1. In Segment, go to Workspace > Add destination > Search for “BigQuery” - it can be found in Storage Destinations

  2. Select BigQuery

  3. Add a name for the destination

  4. Enter your project ID in the Project field

  5. Copy the contents of the credentials (the JSON key) into the Credentials field
    Optional: Enter a region code in the Location field (the default will be “US”)

  6. Click Connect

  7. If Segment can connect with the provided Project ID and Credentials, a warehouse will be created and your first sync should begin shortly

Once the data will reach BigQuery, under your project, you will be able to see a new dataset with multiple tables and partitioned tables (special tables that are divided into segments that make it easier to manage and query your data).