Skip to main content
The distinguishing features of a rollout compared to other types of experiments is that they: The main reasons for gradually rolling something out as opposed to enabling it for all users immediately are:
  • It lets you ensure that the experience you are rolling out is not worsening the user experience before you roll it out to everyone.
  • See comparisons between the group that has the feature you are rolling out and the group that has the current default experience for any metric.
A rollout uses the same basic functionality as an A/B test. It allocates users into two groups, a control group and a treatment group, and compares them. The difference is that in a rollout, you want to roll out a feature gradually. You might start with a 90/10 allocation with 10% of eligible units in the treatment group to limit the impact of a potential issue. You then monitor important metrics to see if you worsen the user experience. If the experiment is going well, you increase the allocation to get more units in the treatment until you have rolled it out completely.

​
Understand Reach

Reach is the share of the total eligible population that receives the treatment. To monitor the rollout, Confidence assigns an additional part of the eligible population to control. With the default 90/10 treatment and control proportions, a rollout at 10% reach assigns 10% of the total eligible population to treatment. Confidence assigns about 1.1% of the total eligible population to control. Monitoring can continue only up to the configured treatment share of the available allocation space, which is 90% by default. Increasing reach above this threshold consumes the comparison group and permanently disables monitoring.

Launch a Rollout

Step-by-step rollout tutorial

Guardrail Metrics

Configure rollout metrics

Monitoring

Monitor rollout health

A/B Tests

Compare with A/B tests