Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Conversion Rate Optimisation › Advanced

Evaluating a Website Relaunch for a Cookware Maker

An agency wants to relaunch an invented cookware maker's site in one day, with new templates, navigation, copy, checkout and tracking together. You break the bundle apart, decide what ships, what can be tested and what can only be judged in a staged rollout, then read the first weeks of results without being fooled by the season, novelty or a counting fault.
ExerciseAdvanced levelModule 2: Conversion work beyond the split test5 minUpdated

Why this matters

A relaunch breaks the intermediate habit of testing every change: many changes arrive together, most pages lack the orders a split needs, and the tracking often changes the same day. The guide to evaluating a website redesign sets the evaluation design and the guide to conversion work without an A/B test sorts what should simply ship; here you apply both under a launch date. If sample-size arithmetic is unfamiliar, revisit the intermediate CRO level.

The cookware maker and its relaunch

Vellerhoe Cookware, an invented cast-iron and enamel cookware maker selling direct online, has an agency that wants everything live on one day in October, before the Christmas peak. Addresses will not change, and the site can serve either set of templates to a visitor assigned at random and keep them on it.

Illustrative example — not a real result

Vellerhoe Cookware: the proposal and the traffic

Proposed changeNote
New product template, all 140 productsGallery, layout, rewritten copy
New navigation and category pagesSite-wide
One-page checkoutReplaces a three-step checkout
New tag set-upPurchase event on a new confirmation page
Zoom fixGallery freezes on some phones
Basket errorsShown only as a red outline
Delivery chargeFirst shown at payment
New homepage photographyCheap to swap back
Page groupVisitors or starts a weekStep measuredVisitors per arm the team's calculation needs
Product template, all products16,000Add to basket, 7.0%22,000
Flagship casserole page alone1,400Add to basket, 9.0%15,000
Checkout650 startsOrder, 62%1,100

Research: tickets often ask whether a pan works on induction, and recordings show buyers checking the "Works on" line under the price before adding to basket. The new copy moves it into a tab.

A bundle of changes with no clean control

A relaunch is a one-way door whose total hides gains and losses that cancel. Make it smaller: ship what needs no question, test what the traffic can answer, and stage the rest against a concurrent comparison group.

Task

  1. Sort each change into ship now, test before launch or relaunch-only, with the reason.
  2. Work out how long each candidate test would take; say which run, and in what order.
  3. List what to record before the build, including what the brief must keep.
  4. Plan the rollout: comparison group, gates, prediction and rollback rules.
  5. Read the results below against the season, novelty and the tracking change, and decide at the gate.
Illustrative example — not a real result

Four weeks at a 50/50 staged rollout, read on the agreed date

MeasureOld siteNew site
Sessions31,00031,000
Orders, shop's order system470505
Orders, analytics471580
Average order value£68£67
Checkout support contacts1923
WeekNew site against old, returning visitorsNew site against old, new visitors
First+14%+6%
Fourth+2%+5%

The agency compares the whole site with the four weeks before launch, in analytics: 1,051 orders on 62,000 sessions against 730 on 52,000, "a 21% lift". Last year the same weeks rose about a fifth on the weeks before. The gate rule: proceed if the new site's orders per session, from the order system, are no more than 5% below the old site's, with guardrails steady.

Worked answer

Sort. Ship now on the current site, without a test: the zoom fix (a bug), the basket errors (a barrier, which the accessibility guide fixes rather than tests) and the delivery charge (a missing cost). The photography is cheap and reversible: ship it with a prediction and a date to check. Test first: the product template and the checkout. Relaunch-only: navigation, category pages and the rest of the design, judged in the staged rollout. The tag set-up is not a design change: install it on the current site first, reconcile it with the order system and log it, so both arms are measured alike.

Illustrative example — not a real result

Weeks each candidate test needs

TestArithmeticWeeks
Product template, all products22,000 × 2 ÷ 16,000about 3
Flagship page alone15,000 × 2 ÷ 1,400about 21
Checkout1,100 × 2 ÷ 650about 3.4

Each row divides the visitors the sample-size calculation asks for, across both arms, by the visitors that page or step receives each week; the flagship page alone would need about five months of its own traffic.

Test the template across all products; the flagship page alone could not finish before the peak. Run the template test, then the checkout test, because both reach the same visitors. The template A/B Test reads add to basket with orders as a Guardrail Metric; the checkout test reads orders directly. Both arms pass a keyboard check first.

Baseline. Orders per session by device and source over full weeks beside the order system's counts; screenshots and proof elements; form fields; campaign destinations; speed; the conversion event. The brief keeps the "Works on" line under the price: research shows it persuading, and a tab would hide it.

Rollout. Random, sticky assignment, identical tracking, widening at gates with a named decision-maker; the prediction and reading date written before launch. Faults are fixed at once; a result below the agreed range restores the specific elements research points to, template by template, not the whole site.

Reading. Tracking first: analytics exceeds the order system in the new arm only, so the purchase event fires twice on the new confirmation page. A fault: fix it, annotate the gap in the measurement change log and read from the order system.

Illustrative example — not a real result

Reading Vellerhoe's four weeks

  • Agency's before-and-after: 1,051 ÷ 62,000 ≈ 1.70% against 730 ÷ 52,000 ≈ 1.40%. Inflated by the double count, and last year's same weeks show the season alone moves the site by about a fifth.
  • Concurrent comparison, order system: 505 ÷ 31,000 ≈ 1.63% against 470 ÷ 31,000 ≈ 1.52%, about 7% higher, on a few hundred orders per arm, so a range running from a small loss to a moderate gain.
  • Returning visitors' lead shrinks, a novelty signature; new visitors, the better guide, hold steadier.
  • Average order value and support contacts barely move.

The concurrent groups shared the season, so it cancels out. The new site's CVR is inside the agreed range with steady guardrails: proceed to the next gate, reported as consistent with a small gain, not a lift of a fifth.

Common mistakes

  • Accepting the one-day launch. Nothing can be attributed.
  • Queuing bug and barrier fixes for the relaunch. They ship now, on the current site.
  • Trusting the before-and-after. It mixes the season and a counting fault into the design.
  • Judging on the first weeks. Novelty fades; read new visitors and wait for the date.
  • Rolling everything back. Restore the specific elements that were lost.

The checkpoint on conversion work without a split test follows.

Pages this lesson applies

Terms used on this page

Also related

Related