> ## Documentation Index
> Fetch the complete documentation index at: https://confidence-auth-testing.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Started with BigQuery

> Configure read and write connections to BigQuery.

![BigQuery logo](https://mintcdn.com/confidence-auth-testing/Cey3NReXEr8kuaqn/images/bigquery-logo.png?fit=max\&auto=format\&n=Cey3NReXEr8kuaqn\&q=85\&s=36d5233ae6e6f1b56346a5618c2ef80c)

## Introduction

> [!NOTE]
>
> If you followed the setup guide on the **Admin** page, your
> setup is already complete. You can skip directly to [what's next](#whats-next).

> [!TIP]
>
> You can run this setup with an AI assistant instead. Run this skill and your assistant guides you through the steps:
>
> ```bash theme={null}
> npx skills add spotify/confidence-ai-plugins --skill setup-warehouse-bigquery
> ```
>
> The installer works with Claude Code, Cursor, Codex, Gemini CLI, and other AI assistants.

This tutorial helps you configure Confidence to:

1. Run queries in BigQuery to compute exposure and metrics.
2. Store assignment data in BigQuery.

Step 2 is optional if you already have assignment data in BigQuery. For
example, if you are using a feature flagging solution other than Confidence
Flags.

This document targets the following audiences:

* Administrators who want to set up Confidence for their organization

> [!NOTE]
>
> In this tutorial, you create a service account in GCP that Confidence impersonates.
> The service account used by Confidence is `account-<your-account-id>@spotify-confidence.iam.gserviceaccount.com`.
> You can name your service account anything you want.
> For clarity, the tutorial assumes you create a service account called `confidence`.
> You can read more about service account impersonation in the [Google Cloud documentation](https://cloud.google.com/iam/docs/service-account-impersonation).

## Before You Begin

* You need to have a Confidence account.
* You need to have a Google Cloud Platform account.
* You need to have permissions to create service accounts.
* You need to have the following APIs enabled in your Google Cloud Platform
  project:
  * BigQuery API
  * Cloud Resource Manager API

## Terraform Scripts

[Terraform scripts are available](https://confidence-cdn.spotifycdn.com/terraform/bq_terraform.tar.gz) to create the service account, set up permissions, and create the required datasets.

If you are using Terraform, then after you have applied the Terraform configuration, you can skip directly to [Step 1c](#step-1c-configure-a-metrics-data-warehouse).

## Step 1: Prepare GCP and Create Data Warehouse Connection

In this step, you:

* create a service account in GCP that Confidence impersonates
* create a dataset for storing the tables that Confidence generates
* configure a metrics data warehouse connection

### Step 1a: Create Service Account

For Confidence to access BigQuery, you need to create a service account that
has permissions to access BigQuery. You also need to allow Confidence to
impersonate the service account. Follow these steps:

**1. Go to Google Cloud Platform Console**

Go to the [Google Cloud Platform Console](https://console.cloud.google.com/) and select the project you want to use for Confidence.

**2. Navigate to Service Accounts**

Go to [IAM & Admin > Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts).

**3. Click + CREATE SERVICE ACCOUNT**

**4. Enter service account details**

Enter a name for the service account, for example, `confidence`. Make note of the service account email address. You can click copy-to-clipboard icon to copy it. Click **CREATE AND CONTINUE**.

**5. Grant roles to the account**

Grant the following roles to the account:

* **BigQuery Job User**
* **BigQuery Data Viewer**

Click **CONTINUE**.

**6. Click DONE**

#### Grant Confidence Access to Impersonate the Service Account

Confidence needs to be able to impersonate the service account you just created
to be able to access BigQuery. You do this by granting one of Confidence's own
service accounts the "Workload Identity User" role, called the principal account:

```text theme={null}
account-<your-account-id>@spotify-confidence.iam.gserviceaccount.com
```

To grant the principal account access to impersonate the service account, follow
these steps:

**1. Open the service account details**

Click on the service account you just created to go to the service account details page.

**2. Select the PRINCIPALS WITH ACCESS tab**

**3. Click GRANT ACCESS**

**4. Enter the principal account**

Enter `account-<your-account-id>@spotify-confidence.iam.gserviceaccount.com` as principal.

**5. Assign the Workload Identity User role**

**6. Click SAVE**

### Step 1b: Create BigQuery Datasets to Store Metrics and Exposure

Confidence needs a dataset in BigQuery where it can store intermediate metrics
and exposure calculations. For Confidence to access the dataset,
you need to grant the principal service account access to the dataset.

```text theme={null}
account-<your-account-id>@spotify-confidence.iam.gserviceaccount.com
```

To create the dataset, follow these steps:

**1. Go to Google Cloud Platform Console**

Go to the [Google Cloud Platform Console](https://console.cloud.google.com/) and select the project you want to use for Confidence.

**2. Create a new dataset**

In [BigQuery](https://console.cloud.google.com/bigquery), click **⋮ > CREATE DATASET** next to the name of the project you want to create it in.

**3. Enter dataset name**

Enter a name for the dataset. For example, `confidence_metrics`.

**4. Select the dataset location**

Select the location where you want to store the dataset.

> [!NOTE]
>
> The dataset location must be the same as the region of your Confidence account. The recommendation is to select a multi-region location for your dataset, like `EU` or `US`.

**5. Click CREATE DATASET**

**6. Click GO TO DATASET**

**7. Click SHARE > Manage Permissions**

**8. Click ADD PRINCIPAL**

**9. Grant permissions**

Enter the email address of the `confidence` service account you created and assign the role **BigQuery Data Owner**.

**10. Click SAVE**

### Step 1c: Configure a Metrics Data Warehouse

With the completion of steps 1a and 1b, you have now created a service account and
a dataset in BigQuery. You can now configure Confidence to use BigQuery to
compute metrics. To do so, follow these steps:

**1. Go to the Confidence app**

**2. Navigate to Metrics Data Warehouse settings**

On the bottom of the left sidebar, select **Admin > Connections > Metrics Data Warehouse**.

**3. Select BigQuery**

**4. Enter connection details**

Enter the GCP Project ID, your metrics dataset (`confidence_metrics`) and the email address of the `confidence` service account you created in step 1a.

**5. Click SAVE**

When you click save, Confidence tries to connect to BigQuery using the
service account you created in step 1a. If you have misconfigured anything,
you should see an error message.

**🎉 Well done! Now you've configured Confidence to compute metrics using BigQuery.**

If you plan to use Confidence Flags, you also need to configure Confidence to
store assignment data in BigQuery. To do so, follow the next steps outlined
below.

## Step 2: Configure Confidence to Write and Read Assignment Data

In this step, you:

* create a dataset in BigQuery that Confidence can write assignment data to
* configure a flag applied connector so that Confidence is able to write assignment data
* configure an assignment table so that Confidence is able to read assignment data

### Step 2a: Create BigQuery Datasets to Store Assignment Data

If you want to use Confidence Flags to run experiments, you need to complete
a few more steps to configure Confidence to store assignment data in BigQuery
and an assignment table that reads from the BigQuery table.

Confidence needs a dataset in BigQuery where it can store assignment data. For
Confidence to access the dataset, you need to grant the principal
service account access to the dataset.

```text theme={null}
account-<your-account-id>@spotify-confidence.iam.gserviceaccount.com
```

To create the dataset, follow these steps:

**1. Go to Google Cloud Platform Console**

Go to the [Google Cloud Platform Console](https://console.cloud.google.com/) and select the project you want to use for Confidence.

**2. Create a new dataset**

Go to [BigQuery](https://console.cloud.google.com/bigquery) and click **⋮ > CREATE DATASET** next to the name of the project you want to create it in.

**3. Enter dataset name**

Enter a name for the dataset, for example, `confidence_flag_applied`.

**4. Select the dataset location**

Select the location where you want to store the dataset.

> [!NOTE]
>
> The dataset location must be the same as the region of your Confidence account. The recommendation is to select a multi-region location for your dataset, like `EU` or `US`.

**5. Click CREATE DATASET**

**6. Click GO TO DATASET**

**7. Click SHARE and then Manage Permissions**

**8. Click ADD PRINCIPAL**

**9. Grant permissions**

Enter the email address of the `confidence` service account you created and add the role **BigQuery Data Owner**.

**10. Click SAVE**

### Step 2b: Configure a Flag Applied Connector

For Confidence to be able to store assignment data in BigQuery, you need to set
up a connector between Confidence and BigQuery.

> [!NOTE]
>
> Assignment data is information on which users were assigned to which variants
> in the experiments you run. Assignment data goes into exposure calculations.
> Metrics use exposure to calculate results in your tests.

This connector is a **"Flag Applied"** connector.
The connector is the part responsible for writing assignment data to BigQuery that Confidence Metrics
can later read.

**1. Go to the Confidence App**

**2. Navigate to Flag applied connections**

On the bottom of the left sidebar, select **Admin > Connections > Flag applied connections**.

**3. Select BigQuery**

**4. Click Next**

**5. Enter connection details**

Enter the Project ID, the dataset (for example, `confidence_flag_applied`) you created in step 2a and the email address of the `confidence` service account you created in step 1a.

**6. Enter the table name**

Enter the name of the table (for example, `flag_applied`) to write assignment data to. Confidence creates the table with the right schema for you.

**7. Click Create**

When you click save, Confidence tries to connect to BigQuery using the
service account you created in step 1a. If you have misconfigured anything,
you should see an error message.

### Step 2c: Configure an Assignment Table

For Confidence to use the stored assignment table, you need to set
up an assignment table that reads from the BigQuery table. You first need to create
an entity, which represents the thing you're experimenting on, like your users.
To do so, follow these steps:

**1. Go to the Confidence App**

**2. Navigate to the BigQuery connection**

On the bottom of the left sidebar, select **Admin > Connections > Flag applied connections** and then the BigQuery connection you created.

**3. Click Create in the Assignment table section**

**4. Enter assignment table name**

Enter a name for the assignment table, such as `flag_applied`. This name should typically match the name you used in step 2b. Confidence can then read assignments from the destination table of your flag assignments.

**5. Create or select an entity**

Create a new entity or select an existing entity. Entities are the things you're experimenting on, like your users. Enter `User` and specify the data type of the identifier that identifies the entity. For example, if you have a UUID that identifies your users, your primary key type is a **String**.

**6. Click Create**

**🎉 Well done! You are all set up and ready to go.**

## Step 3: Grant View Access to Metrics Data Sources

Confidence uses fact tables to describe the data that you create metrics from.
To ensure that Confidence can access data from the data sources you use for fact tables, you
need to grant the role **BigQuery Data Viewer** to the service account named `confidence` that you have created.
Grant this role for the datasets and tables you want Confidence to have access to.

**1. Go to Google Cloud Platform Console**

Go to the [Google Cloud Platform Console](https://console.cloud.google.com/) and select the project that the data you want to grant Confidence access to is in.

**2. Select the dataset or table**

Go to [BigQuery](https://console.cloud.google.com/bigquery) and select the dataset. Optionally select the table.

**3. Open permissions dialog**

If you selected a dataset, click **SHARING > Manage Permissions**. If you selected a table, click **SHARE**.

**4. Click ADD PRINCIPAL**

**5. Grant permissions**

Enter the email address of the `confidence` service account you created and assign the role **BigQuery Data Viewer**.

**6. Click SAVE**

## What's Next?

The next step is to [create a fact table](/docs/capabilities/metrics/fact-tables.md), and a
[metric](/docs/capabilities/metrics/overview). For an overview, see the [metric introduction](/docs/capabilities/metrics/overview)
page, and the [metrics quickstart](/docs/guides/create-your-first-metric.md).

## Related Resources

> **[Metrics Introduction](/docs/capabilities/metrics/overview)**
>
> Overview of metrics in Confidence

> **[Fact Tables](/docs/capabilities/metrics/fact-tables.md)**
>
> Configure fact tables for metrics

> **[Assignment Tables](/docs/capabilities/metrics/assignment-tables.md)**
>
> Set up assignment tracking

> **[Configure Metric Quickstart](/docs/guides/create-your-first-metric.md)**
>
> Create your first metric