Configure Variance Reduction - Confidence Documentation

Variance reduction uses pre-exposure data to reduce noise in your experiment metrics. This allows you to detect effects with fewer samples or detect smaller effects with the same sample size.

How It Works

The Stats API applies regression adjustment to reduce variance. You provide pre-exposure measurements alongside your post-exposure data, and the API uses the correlation between them to produce more precise estimates.

Configure Variance Reduction for Analysis

Include pre-exposure data in the adjustment field within your summary data:

{
  "id": "my-metric",
  "segments": [
    {
      "dimensions": {},
      "groups": [
        {
          "group": "control",
          "data": {
            "zTest": {
              "summary": {
                "mean": 10.5,
                "variance": 25.0,
                "count": 1000,
                "adjustment": {
                  "mean": 9.8,
                  "variance": 22.0,
                  "covariance": 15.2
                }
              }
            }
          }
        },
        {
          "group": "treatment",
          "data": {
            "zTest": {
              "summary": {
                "mean": 11.2,
                "variance": 26.0,
                "count": 1000,
                "adjustment": {
                  "mean": 9.9,
                  "variance": 21.5,
                  "covariance": 14.8
                }
              }
            }
          }
        }
      ]
    }
  ]
}

Adjustment Fields

The adjustment object contains:

Field	Description
`mean`	Mean of the pre-exposure measurements
`variance`	Variance of the pre-exposure measurements
`covariance`	Covariance between pre-exposure and post-exposure values

Configure Variance Reduction for Power Analysis

For power analysis, include the expected covariance adjustment in your power data:

{
  "hypotheses": [
    {
      "id": "my-metric",
      "segments": [
        {
          "dimensions": {},
          "powerData": {
            "zTest": {
              "baselineMean": 10.0,
              "baselineVariance": 25.0,
              "adjustment": {
                "baselineVariance": 22.0,
                "baselineCovariance": 15.0
              }
            }
          }
        }
      ]
    }
  ]
}

Power Analysis Adjustment Fields

Field	Description
`baselineVariance`	Variance of the pre-exposure covariate
`baselineCovariance`	Covariance between the pre-exposure covariate and the outcome

Interpret Results

The API returns both adjusted and unadjusted estimates. The treatment effect estimate benefits from the reduced variance, providing tighter confidence intervals. The response includes a varianceReductionRate field showing the achieved reduction.

Variance Reduction

Learn about variance reduction concepts

Run an Analysis

Complete analysis tutorial

Run Power Analysis

Calculate required sample sizes

​How It Works

​Configure Variance Reduction for Analysis

​Adjustment Fields

​Configure Variance Reduction for Power Analysis

​Power Analysis Adjustment Fields

​Interpret Results

​Related Resources