Definition
A staged rollout is the release of a change in steps to a growing share of visitors, pages or regions, with a reading at each step deciding whether the next goes ahead. The guide to conversion work without A/B tests puts it in the middle ground between a full test and a simple before-and-after reading: the change reaches more people over time, and it counts as a controlled comparison only when the groups are drawn at random and run at the same time.
Two jobs hide inside the term. One is caution: a fault in a new template or site touches a fraction of visitors or pages before it is noticed, and a pause or a rollback limits the harm. The other is evidence: if the share on the new version is chosen by chance and the old version keeps running beside it, the two groups can be compared directly. A rollout can do the first without the second, and the write-up should say which it did.
When a rollout is evidence and when it is only caution
The guide to evaluating a redesign sets out the difference, which lies in how the share is chosen and when each part of it runs.
- Random and concurrent. Visitors, or pages, are assigned to the new version by chance, each visitor stays in the same group on every visit, both versions are measured with identical tracking, and both run in the same weeks. Season, campaigns and the traffic mix then fall on both groups alike, so a gap between them can be put down to the change.
- Chosen and sequential. The new version goes to mobile visitors first, to one country first, to the strongest pages first, or to everyone on a fixed date. Each step is then compared with a different period or a different kind of visitor, and whatever else changed in those weeks is mixed into the reading.
The second kind is still sound release discipline. Read it the way the comparison of control charts and A/B testing reads any change that has already shipped: control limits set from the metric's past values make the reading more disciplined, but a shift is a reason to investigate rather than proof of cause. Either way, the result goes in the learning log, labelled with the strength of its evidence.
In practice: site changes, page sets and business cases
Site changes. Changes too large to split cleanly, such as a rebuilt template or a new site, are where a rollout earns its place: the redesign guide widens the new site's share only at gates agreed before launch, each with a named decision-maker and criteria written in advance.
Page sets. The programmatic SEO guide publishes a templated page set in tranches: a pilot of the strongest pages read by hand, a few weeks of checking crawling, indexing and use, and the next tranche only if the pilot earns it. Tranches grow by the rows that meet the data minimum, not by a target count. The SaaS SEO guide treats integration and template directories as page sets of this kind. Because these tranches are chosen rather than random, each gate asks whether the pages are indexed and used, not what the template caused.
Business cases. The tutorial on building a marketing business case asks, before approval, how anyone will know the investment worked, and names a staged rollout beside a holdout and a regional comparison as ways to show it. That holds for a rollout whose groups are random and concurrent, agreed before the money is spent.
A recipe publisher moves one category in random batches
Zolduax Recipes, an invented recipe publisher, built a new page template and chose its baking category, 1,600 recipes, to move first. It split the category at random into 4 batches of 400 and released one batch at a time, with 2 weeks between them. Because the batches were drawn by chance and both templates ran in the same weeks, the recipes still on the old template served as the comparison. After batch 1, Search Console showed the new pages indexed as before, but readers reached the method on 9% fewer visits than on old-template recipes: on phones, the new ingredient panel pushed it below a second advert. The panel was shortened before batch 2, the gap closed to within the usual week-to-week spread, and batches 3 and 4 followed on the planned dates.
Related terms
An A/B TestControlled comparison of two variants. is the controlled comparison a random, concurrent rollout imitates: two versions, random assignment, the same period and a decision rule fixed in advance. A Holdout GroupA randomly chosen part of an audience kept from exposure to measure what exposure caused. works the other way round, keeping a small random group on the old experience after a change ships to measure its cumulative effect, at the cost of that group missing the benefit. IncrementalityConversions caused by marketing. is what a well-built rollout can show and a sequential one cannot. A Geo ExperimentA test that runs a campaign in some regions and holds it back in comparable ones, then compares total outcomes, where people cannot be randomised one by one. applies the same logic where people cannot be randomised one by one: some regions receive a campaign and comparable ones are held back.
Common confusions
- Every staged rollout is a test. Only random, concurrent groups give a comparison. Phones first or one country first is a sequence of before-and-after readings.
- A rollout is as good as a test where traffic allows one. Where the change can be divided and there is enough traffic to read a test, the A/B test remains the default; a rollout is for changes that have to ship without a split.
- Publishing in tranches proves the template works. It shows whether each tranche is indexed and used before the next is released; it does not compare the template with an alternative.
- Launching a campaign market by market is a staged rollout. The term covers releasing one change to a growing share of visitors, pages or regions, not sequencing a campaign or rebrand across markets.
