Why a split test changes how the inputs are used
The default calculator page imagines one page, one period and an optimistic question: what if this rate were higher? An A/B TestControlled comparison of two variants. asks something narrower. Two versions of the page receive randomly split traffic, and the team wants to know whether one converts better by enough to matter. That reframes all three inputs at once.
Conversions and visitors stop describing the whole page and start describing one arm. The control gets its own run and the variant gets another, and the two only make sense side by side if they cover the same calendar days and count visitors with the same rule. Testing tools assign a person to an arm and keep them there, so the visitor figure should come from the tool's assignment counts rather than from a general analytics report that may count sessions instead.
The uplift slider changes character most of all. On the default page it is a "what if" dial, and it is tempting to drag it to a satisfying number. For a test it should hold the minimum detectable effect written into the plan. That value decides how much traffic the test needs and how long it must run, which the sample size and significance guide works through. Putting it into the calculator converts an abstract percentage into a count of extra enquiries or orders, which is the figure a manager can weigh against the cost of building the variant.
The inputs for a landing page test
| Calculator input | On the default page | For a landing page test | Where to find it |
|---|---|---|---|
| Conversions | Conversions for the page in the period | Conversions for the control, then for the variant, over the test dates | Testing tool or analytics, reported per variant |
| Visitors / sessions | All visitors or sessions | Visitors bucketed into that same variant, counted the way the testing tool assigns them | Testing tool assignment counts |
| Relative CVR improvement | Any improvement worth imagining | The minimum detectable effect the test is powered for | Test plan and the sample-size calculation behind it |
The changed input is derived, not chosen on the day. Start from the control's conversion rate before the test, decide the smallest relative lift that would justify shipping the change, and check with a sample-size method that the page receives enough traffic to detect it within the time you can afford. If it does not, the honest move is to raise the effect you are prepared to detect or to test somewhere busier, not to keep the small figure and hope. A well-formed test hypothesis usually names this effect alongside the reason the change should produce it.
Entering the test inputs step by step
Measure the control before the test starts
Take a recent, stable window of traffic to the control page and enter its conversions and visitors. The rate this returns is the baseline every later calculation leans on, so avoid windows containing promotions, outages or unusual campaigns.
Set the uplift to the planned minimum effect
Move the uplift input to the minimum detectable effect from the plan. Read the extra conversions it implies and decide whether that gain covers the design, build and opportunity cost of the test. If it does not, rethink the test before it launches. The opportunity cost framework frames that cost as the best other use of the same traffic and build time, such as a test on a busier page.
Run each arm separately once data arrives
When the test has run, enter the control's conversions and assigned visitors, note the rate, then repeat for the variant. Keep the uplift fixed at the planned value so the extra-conversions line stays comparable between runs.
Hand the counts to the significance method
Take the two sets of counts to whatever significance or sequential method the team uses. The calculator has described each arm; deciding whether the gap is real belongs to that method.
A worked quote-page test with illustrative inputs
A heating installer sizes a quote-page test
Flueperch Heating, an invented boiler-installation firm, wants to test a shorter quote page. Over a recent four-week window the control page converted 168 quote requests from 4,200 assigned visitors, a conversion rate of 4.0%. The team's test plan sets a minimum detectable effect of a 12% relative uplift, which would take the rate to roughly 4.5%. Entered into the calculator, that uplift is worth about 20 extra quote requests for each equivalent four-week period. The firm judges that twenty more requests a month justify the build effort, so the test goes ahead, with the duration taken from a separate sample-size calculation rather than from this tool.
Conversion Rate Calculator
Open the Conversion Rate Calculator →Prefilled with the example's inputs: Conversions 168 · Visitors / sessions 4,200 · Relative CVR improvement 12% → Conversion rate 4% · Extra conversions from uplift 20.
All figures belong to the invented firm. Try lowering the uplift in the embedded calculator to see how quickly the prize shrinks, which is often the moment a team realises the change it wanted to test is too small to notice.
Interpreting the calculator's outputs for a test
Read the conversion rate for the control first, because it is the baseline the whole test is planned around. Every minimum effect is relative to it, and a baseline taken from an odd week will make the planned effect either too easy or too hard to detect. If the pre-test rate wanders a lot from week to week, that wandering is itself a warning that the test will need more traffic than a quick estimate suggests.
Next, read extra conversions from the uplift as the value at stake. It answers the planning question of whether the test deserves the page at all. A small prize on a low-traffic page often means the effort would be better spent on a page further up the FunnelModel of the customer journey stages. where more visitors pass through.
Once the test has run, the two rates you get from separate runs describe what each arm did. Their difference is the observed effect, and it is only an observation. Early in a test the gap swings widely, and a variant that leads after a few days can easily fall back. The calculator has no notion of that uncertainty.
Significance and sample size therefore come from the testing method, not from this page. The A/B testing guide sets out how the comparison should be run, and the landing page headline experiment shows a test plan written this way, with the effect and duration fixed before launch. What the calculator adds is translation: a relative lift becomes a count of enquiries, orders or sign-ups that people outside the testing team can reason about.
Pitfalls when test numbers go into the calculator
- Entering an optimistic uplift instead of the minimum detectable effect. A large hoped-for lift makes every idea look worth testing and hides the fact that the realistic effect may be too small to detect.
- Stopping when the variant first looks ahead. Repeatedly checking two separate runs and stopping at the first favourable gap inflates false positives; let the planned sample complete.
- Comparing variants over different dates. Running the control on last month and the variant on this month turns seasonality and campaign changes into a fake effect.
- Counting sessions in one variant and users in the other. A returning visitor creates several sessions, so mixing the two definitions shifts one arm's rate for reasons unrelated to the page.
- Reading the extra conversions as a forecast. The figure assumes the effect is exactly the planned one; the real outcome may be larger, smaller or nothing at all.
