Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Pricing Page Layout Test

A test design for a pricing page layout: a hypothesis on plan presentation, plan starts per visitor as the metric, revenue guardrails and reading results.
Experimentintermediate6 min readUpdated

Hypothesis

Presenting plans with a recommended option and a plain comparison of what each includes will raise paid plan starts per pricing page visitor, because buyers can choose without asking sales. The mechanism is decision effort. A pricing page that lists features in marketing language, with no indication of which plan suits whom, leaves buyers to work out the difference themselves, and many defer the decision or send a question to sales instead.

Evidence for this hypothesis typically includes sales or support logs full of "which plan do I need?" questions, recordings that show visitors moving back and forth between plan columns, and usability research such as the Nielsen Norman Group articles on comparison tables. Before committing traffic, score the idea against other candidates with the test prioritisation guide; a pricing page is often high-traffic and high-value, but the test is slow because the outcome is a purchase. Record the hypothesis, the evidence and the exact layout change in the experiment hypothesis template.

Metrics and guardrails

The primary metric is plan starts per pricing page visitor: visitors in each arm who begin a paid plan, whether by checkout or by starting a paid trial that requires payment details, divided by visitors who viewed the pricing page in that arm. Count starts from the billing system, not the analytics tool, so that abandoned checkouts are not counted as starts. The conversion rate calculator gives you each arm's CVR from those counts.

Average revenue per account is the first guardrail. A recommended plan works by steering choice, and if it steers buyers towards a cheaper plan than they would have picked, starts can rise while revenue falls. Refund and downgrade rate is the second: a clearer layout should help people choose the right plan, so a jump in early refunds or downgrades suggests the page is pushing people into the wrong one. Over a longer horizon, compare LTV by arm once enough time has passed.

Design

Randomise at visitor level and hold the assignment across sessions and devices where your tool can, since buyers rarely purchase on their first visit to a pricing page. The control is the current layout. The challenger changes presentation only: one plan marked as recommended for a stated type of buyer, and a plain-language comparison of what each plan includes. Prices, plan names, billing periods and discounts stay identical in both arms.

Design elementChoiceReason
Unit of randomisationVisitor, held across sessionsBuyers return several times before deciding
VariantsCurrent layout versus recommended plan with plain comparisonOne presentation change, prices untouched
Primary count sourceBilling systemExcludes abandoned checkouts
Sales-assisted dealsTagged and reported separatelyDifferent decision process and timeline
DurationAt least one typical buying cycle, in whole weeksLets returning visitors complete their decision
ExclusionsExisting customers viewing pricing to upgradeThey follow a different path

Duration follows the buying cycle rather than traffic alone. If most buyers decide within a few days of first viewing pricing, a few weeks may suffice; if they deliberate for longer, the test must run long enough for visitors assigned early to finish deciding.

Sample size reasoning

Paid plan starts are usually a small share of pricing page visitors, so the sample needed is large compared with a test on a click or a sign-up. The inputs are the current start rate, the smallest lift worth shipping and the confidence you require; the sample size and significance guide shows how they combine. Because the guardrail on revenue per account is also a comparison between arms, check that the planned sample is large enough to spot a meaningful drop there too, not only a rise in starts.

If traffic is too thin, resist the temptation to switch to a proxy such as clicks on a plan button. Those clicks are easier to move than purchases and can mislead. A better option is a bolder layout change, a longer run, or accepting that the test can only confirm a large effect.

Reading the results

Read once, after the planned sample and a full buying cycle have both been reached. Start with plan starts per visitor and decide whether the difference exceeds noise. Then look at the plan mix in each arm: which plans did buyers choose? A win that comes entirely from a shift towards the recommended plan is still a win if revenue per account holds, but you need to know that is the mechanism.

Next check the guardrails. If revenue per account fell, the recommended plan may be too cheap a default; try recommending a different plan rather than abandoning the layout. If refunds and downgrades rose, the comparison may be unclear about limits that matter. Finally, look at the sales-assisted segment separately. A layout that reduces "which plan" questions to sales without hurting self-serve starts is a useful result even if the primary metric is flat.

Illustrative example — not a real result

A scheduling app adds a recommended plan

Rotaly, an invented staff scheduling app, has three plans shown as identical columns. The team tests a layout that marks the middle plan as "Best for teams of 10 to 50" and adds a plain list of what each plan includes. Over five weeks, with about 6,500 pricing page visitors per arm, plan starts per visitor rise from 3.0% to 3.6%. The share choosing the middle plan rises, and average revenue per account stays within a few pounds of the control. Refunds in the first month show no change. "Which plan?" emails to sales fall noticeably, which the team logs as a secondary benefit.

Pitfalls

Plan mix shifts are the defining pitfall of pricing tests. A layout can change which plan people choose without changing whether they buy, and that can be good or bad for revenue depending on the direction. Always report the mix alongside the primary metric.

Returning visitors are the second. If assignment is not held across sessions, a buyer may see one layout on Monday and another on Thursday, and their purchase cannot be credited to either. The test pitfalls guide covers this sample pollution in general terms. Sales-assisted deals are the third: they arrive weeks later, often after a call, and the page played only a small part. Mixing them into the primary metric adds slow, noisy data. Finally, avoid running the test across a price change, a promotion or a plan launch; any of those makes the arms incomparable. The pricing and packaging guide is where price and plan changes belong, and they should be scheduled around this test.

The rest of the page that this test holds constant, from complete costs and billing periods to a call to action for each plan, is covered in the guide to designing a pricing page.

Common mistakes

Pricing pages carry more commercial risk than most pages, so mistakes here are costly. If you sell physical goods rather than plans, the free shipping threshold test applies the same margin-first thinking to delivery offers, and the experiment library lists every other design.

  • Testing prices instead of presentation. This design changes layout only; price tests need a different, carefully governed approach.
  • Using plan button clicks as the metric. Clicks are cheap; starts and revenue are what matter.
  • Ignoring the plan mix. A rise in starts towards a cheaper plan can cost revenue.
  • Reading before buyers finish deciding. Visitors assigned late have not had time to buy.
  • Letting existing customers into the test. Upgraders behave differently and dilute the result.
  • Launching during a promotion. Discounts swamp layout effects and make the arms unequal.

Frequently asked questions

Can we test different prices with this design?

No. This design changes only how the existing plans and prices are presented. Showing different prices to different visitors raises fairness and trust questions and belongs to pricing strategy, which the product marketing guide covers.

Why is average revenue per account a guardrail?

A recommended plan nudges buyers towards one option. If that option is cheaper than what they would otherwise have chosen, starts can rise while revenue falls, so the guardrail checks that the mix has not shifted downwards.

What about visitors who talk to sales before buying?

Track them separately. Sales-assisted deals take longer and are often decided on things the page cannot show, so mixing them into the primary metric adds noise. Report them as a secondary read.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Part of these learning paths

Also related

Sources

  1. Nielsen Norman Group articles — Nielsen Norman Group

Related