Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Conversion Rate Optimisation › Intermediate

Capstone: An Experimentation Roadmap for an Online Store

The capstone of the intermediate CRO level asks you to turn an invented online optician's research pack into a working experimentation programme for a quarter. It draws on prioritisation, test design, sample size, checkout and product page friction and programme governance, and is judged against explicit criteria.
Capstone projectIntermediate levelModule 3: Checkout, product pages and programmes7 minUpdated

Brief

Brackenford Opticians is an invented independent optician that sells prescription glasses online alongside its single high-street practice. Shoppers choose a frame, configure lenses, enter their prescription, and buy; a free home try-on service sends a few frames to try before buying. The owner is the practice's optometrist. An operations manager runs dispatch and returns, a marketing coordinator runs the site and email, and a freelance developer works on the site a few days a week. Nobody owns testing.

A testing tool was installed last year. Since then the team has run a handful of tests, each chosen by whoever asked loudest, two stopped early when they "looked good", and nobody can say what was learned. The owner has hired you to set up a programme for the next quarter and to leave it running when you go.

The constraints:

  • The developer's time is the scarcest resource, and every test competes with routine site work.
  • The owner insists that shoppers create an account so their prescription is kept on record. Changing that needs her sign-off after she checks her professional obligations, which she will do this quarter but not this month.
  • The annual sale runs late in the quarter. Nothing should be tested on affected pages during it.
  • The testing tool is client-side, so flicker is a live risk on heavier pages.
  • The owner reads one page a month and wants to know what changed and what was learned, not how many tests ran.

The store and its data

Illustrative example — not a real result

Brackenford's monthly funnel

StepShoppersStep rate
Sessions70,000
Frame product page visitors38,000
Lens configurator started4,20011.1%
Prescription entered2,50059.5%
Added to basket2,30092.0%
Checkout started1,50065.2%
Orders1,05070.0%

Average order value: £118. Home try-on requests: 600 a month. Returns: 9% of orders, most often because the frame was too wide or too narrow. The quarter is 13 weeks; the sale takes weeks 10 and 11. No single test may run longer than six weeks.

Illustrative example — not a real result

The research pack

IDFindingEvidence
R-aShoppers stall at the pupillary distance field on the prescription step; many leave thereRecordings; support emails asking how to measure it
R-bShoppers leaving the lens step say they did not know which lenses they neededExit poll; recordings of repeated switching between options
R-cFrame width and bridge measurements sit in a closed tab; fit is the top return reasonReturns data; recordings
R-dGlasses are made to order and arrive in 7 to 10 days, first stated at checkoutWalkthrough; "where is my order" emails
R-eAn account is required before paymentWalkthrough; funnel data
R-fReviews praise home try-on, which is linked only from the footerReviews
R-gThe owner would like a video on the homepageNone
R-hA vendor offers a virtual try-on toolThe vendor's own claims

Task

Produce a roadmap document with the parts below, in this order. Use the experimentation programme playbook as the backbone: intake, prioritisation, design and QA, run and monitor, read-out and learning log.

  1. Intake. Turn every finding into a hypothesis card on the experiment hypothesis template, naming the observation, the change, the expected effect, the primary metric and the guardrails. Findings with no evidence go back with a recorded reason; say what research would earn them a place.
  2. Feasibility. For each hypothesis, record the page or step, its eligible traffic, its baseline CVR from the funnel (the conversion rate calculator turns counts into rates), the smallest effect worth shipping, the visitors per arm and the run length in whole weeks. Follow the reasoning in the guide to sample size and significance, and park, redesign or move any hypothesis that breaks the ceiling.
  3. Scoring and sequence. Score what survives on a scale you state in advance, counting evidence sources rather than rating confidence, as the guide to prioritising tests recommends. Tag each by page and step, turn shared tags into sequences, and record dependencies with the event that clears them.
  4. Fixes without a test. Identify findings better answered by shipping a change and watching the step rate, and explain why each does not need a test slot.
  5. Two full test plans. For the first two tests, write the hypothesis, unit of randomisation, allocation, primary metric, guardrails (including AOV and the return rate where relevant), sample, duration, stopping rule, QA steps including a flicker check, and the decision you will take for each possible result. The A/B testing guide sets out the procedure.
  6. The calendar. Lay the tests and fixes across the quarter, respecting the ceiling, the sale, the account dependency and the developer's capacity, with slack for overruns.
  7. Roles and cadence. Assign the playbook's roles to Brackenford's people, including who alone declares a result, and set the weekly stand-up, fortnightly read-out and quarterly review.
  8. Reporting. Design the learning log fields and the one-page monthly read-out for the owner, stating results as ranges and recording losses and inconclusive tests with the same care as wins.

Draw on the level's friction work as you write the hypotheses: the guide to checkout optimisation for the account and delivery findings, and the guide to product page optimisation for frame measurements, fit and home try-on. The experimentation loop framework is the check that every test ends in a write-up that feeds the next round of research.

Illustrative example — not a real result

One row of a strong feasibility table, for comparison

HypothesisStep and eligible trafficBaselineSmallest effectVisitors per armRun length
A "how to measure" helper beside the pupillary distance field, with an option to add it later, raises the prescription step ratePrescription step, 4,200 configurator starts a month, about 970 a week59.5%10% relative, to 65.5%About 1,0302,060 ÷ 970 = 2.1 weeks, so 3 whole weeks

The row shows its arithmetic, rounds up to whole weeks, and ties the hypothesis to two independent sources from the research pack.

Evaluation criteria

Assess your roadmap against each row, or ask a peer to. A strong roadmap meets every row; most first attempts miss two or three.

CriterionWhat a strong answer showsCommon gap
Evidence at intakeEvery hypothesis traces to a named finding and source; the video and the vendor's tool are returned with the research that would justify themUnevidenced ideas kept because of who proposed them
Feasibility firstRun lengths computed for every hypothesis and applied before scores are read, with a decision for each that breaks the ceilingScores sorted first, feasibility discovered after launch
Consistent scoringA stated scale, evidence counted, effort from the developerScores with no rationale, or everything rated high
Sequencing and dependenciesTests on the same page or adjacent steps sequenced; the account change held for the owner's sign-off; nothing scheduled on sale pages during the saleOverlapping tests on the lens and prescription steps
Fixes without testsLow-risk, well-evidenced changes such as surfacing frame measurements shipped and monitored, freeing test slotsTest slots spent confirming obvious fixes
Test plansUnit, metric, guardrails, sample, duration, stopping rule and a pre-agreed decision for each outcome, plus a flicker checkA plan with no sample size or no guardrails
Programme governanceNamed roles with one person alone declaring results, and a cadence that keeps health checks apart from read-outsChannel owners calling their own winners
Honest reportingResults reported as ranges; losses and inconclusive tests logged; an "implemented?" field trackedA monthly page listing winners only
Fit to the businessThe calendar respects the developer's capacity and leaves slack; the owner's page answers what changed and what was learnedA roadmap that needs a full-time team

Submission checklist

0/12

When you have finished, compare your programme with the phases in the experimentation programme playbook, then return to the intermediate CRO level to mark the level complete. Your plans should treat every A/B Test as one turn of a loop that keeps running after you leave.

Pages this lesson applies

Terms used on this page

Also related

Related