Hypothesis
Presenting plans with a recommended option and a plain comparison of what each includes will raise paid plan starts per pricing page visitor, because buyers can choose without asking sales. The mechanism is decision effort. A pricing page that lists features in marketing language, with no indication of which plan suits whom, leaves buyers to work out the difference themselves, and many defer the decision or send a question to sales instead.
Evidence for this hypothesis typically includes sales or support logs full of "which plan do I need?" questions, recordings that show visitors moving back and forth between plan columns, and usability research such as the Nielsen Norman Group articles on comparison tables. Before committing traffic, score the idea against other candidates with the test prioritisation guide; a pricing page is often high-traffic and high-value, but the test is slow because the outcome is a purchase. Record the hypothesis, the evidence and the exact layout change in the experiment hypothesis template.
Metrics and guardrails
The primary metric is plan starts per pricing page visitor: visitors in each arm who begin a paid plan, whether by checkout or by starting a paid trial that requires payment details, divided by visitors who viewed the pricing page in that arm. Count starts from the billing system, not the analytics tool, so that abandoned checkouts are not counted as starts. The conversion rate calculator gives you each arm's CVRShare of visits or clicks that convert. from those counts.
Average revenue per account is the first guardrail. A recommended plan works by steering choice, and if it steers buyers towards a cheaper plan than they would have picked, starts can rise while revenue falls. Refund and downgrade rate is the second: a clearer layout should help people choose the right plan, so a jump in early refunds or downgrades suggests the page is pushing people into the wrong one. Over a longer horizon, compare LTVTotal value a customer brings over time. by arm once enough time has passed.
Design
Randomise at visitor level and hold the assignment across sessions and devices where your tool can, since buyers rarely purchase on their first visit to a pricing page. The control is the current layout. The challenger changes presentation only: one plan marked as recommended for a stated type of buyer, and a plain-language comparison of what each plan includes. Prices, plan names, billing periods and discounts stay identical in both arms.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Visitor, held across sessions | Buyers return several times before deciding |
| Variants | Current layout versus recommended plan with plain comparison | One presentation change, prices untouched |
| Primary count source | Billing system | Excludes abandoned checkouts |
| Sales-assisted deals | Tagged and reported separately | Different decision process and timeline |
| Duration | At least one typical buying cycle, in whole weeks | Lets returning visitors complete their decision |
| Exclusions | Existing customers viewing pricing to upgrade | They follow a different path |
Duration follows the buying cycle rather than traffic alone. If most buyers decide within a few days of first viewing pricing, a few weeks may suffice; if they deliberate for longer, the test must run long enough for visitors assigned early to finish deciding.
Sample size reasoning
Paid plan starts are usually a small share of pricing page visitors, so the sample needed is large compared with a test on a click or a sign-up. The inputs are the current start rate, the smallest lift worth shipping and the confidence you require; the sample size and significance guide shows how they combine. Because the guardrail on revenue per account is also a comparison between arms, check that the planned sample is large enough to spot a meaningful drop there too, not only a rise in starts.
If traffic is too thin, resist the temptation to switch to a proxy such as clicks on a plan button. Those clicks are easier to move than purchases and can mislead. A better option is a bolder layout change, a longer run, or accepting that the test can only confirm a large effect.
Reading the results
Read once, after the planned sample and a full buying cycle have both been reached. Start with plan starts per visitor and decide whether the difference exceeds noise. Then look at the plan mix in each arm: which plans did buyers choose? A win that comes entirely from a shift towards the recommended plan is still a win if revenue per account holds, but you need to know that is the mechanism.
Next check the guardrails. If revenue per account fell, the recommended plan may be too cheap a default; try recommending a different plan rather than abandoning the layout. If refunds and downgrades rose, the comparison may be unclear about limits that matter. Finally, look at the sales-assisted segment separately. A layout that reduces "which plan" questions to sales without hurting self-serve starts is a useful result even if the primary metric is flat.
A scheduling app adds a recommended plan
Rotaly, an invented staff scheduling app, has three plans shown as identical columns. The team tests a layout that marks the middle plan as "Best for teams of 10 to 50" and adds a plain list of what each plan includes. Over five weeks, with about 6,500 pricing page visitors per arm, plan starts per visitor rise from 3.0% to 3.6%. The share choosing the middle plan rises, and average revenue per account stays within a few pounds of the control. Refunds in the first month show no change. "Which plan?" emails to sales fall noticeably, which the team logs as a secondary benefit.
Pitfalls
Plan mix shiftsA change in a blended metric caused by a change in the mix of segments, such as devices or sources, rather than by any segment performing differently. are the defining pitfall of pricing tests. A layout can change which plan people choose without changing whether they buy, and that can be good or bad for revenue depending on the direction. Always report the mix alongside the primary metric.
Returning visitors are the second. If assignment is not held across sessions, a buyer may see one layout on Monday and another on Thursday, and their purchase cannot be credited to either. The test pitfalls guide covers this sample pollution in general terms. Sales-assisted deals are the third: they arrive weeks later, often after a call, and the page played only a small part. Mixing them into the primary metric adds slow, noisy data. Finally, avoid running the test across a price change, a promotion or a plan launch; any of those makes the arms incomparable. The pricing and packaging guide is where price and plan changes belong, and they should be scheduled around this test.
The rest of the page that this test holds constant, from complete costs and billing periods to a call to action for each plan, is covered in the guide to designing a pricing page.
Common mistakes
Pricing pages carry more commercial risk than most pages, so mistakes here are costly. If you sell physical goods rather than plans, the free shipping threshold test applies the same margin-first thinking to delivery offers, and the experiment library lists every other design.
- Testing prices instead of presentation. This design changes layout only; price tests need a different, carefully governed approach.
- Using plan button clicks as the metric. Clicks are cheap; starts and revenue are what matter.
- Ignoring the plan mix. A rise in starts towards a cheaper plan can cost revenue.
- Reading before buyers finish deciding. Visitors assigned late have not had time to buy.
- Letting existing customers into the test. Upgraders behave differently and dilute the result.
- Launching during a promotion. Discounts swamp layout effects and make the arms unequal.
