Why this matters
The intermediate level started from a backlog that already existed. Here you decide what research produces it, and most programmes fail at this point: recordings watched without a question, two funnels at once, nobody to decide. The tutorial on running a conversion research sprint gives the order: question and owner, tracking check, locate the leak, then watch, ask, observe and read at that one step, and triangulate before writing rows. The usability testing tutorial covers the method most often run badly. If prioritisation and test design are not yet familiar, revisit the intermediate CRO level first.
The framer and its configurator
Framwick Picture Framing is an invented online framer. Visitors choose a moulding, enter the size of their artwork, pick a mount and glazing, see a preview with the price and add the frame to their basket. Phone visitors start the configurator more often than desktop visitors and finish it far less often. The owner wants to know why, and what to change first.
Framwick Picture Framing: the configurator over four weeks
| Step | Phone, before the tracking fix | Phone, after the fix | Desktop |
|---|---|---|---|
| Configurator started | 5,200 | 5,200 | 3,100 |
| Artwork size entered | 2,300 | 2,300 | 2,600 |
| Mount chosen | 2,050 | 2,050 | 2,350 |
| Glazing chosen | 900 | 1,950 | 2,250 |
| Added to basket | 1,700 | 1,700 | 2,000 |
- The tracking check found that the glazing event did not fire from the phone layout's "Next" button, which is a different element from the desktop button. It was fixed in the first days of the sprint.
- On phones, 55% of baskets became orders in the period.
What the methods returned once aimed at the located step:
| Source | Finding |
|---|---|
| Recordings, phone visitors who left at the size step | In 14 of 40 sessions the visitor typed a size, switched between centimetres and inches, and the field cleared. In 11 the visitor opened the size guide image, zoomed, then left. The team's rule, set before watching, was that a behaviour counts if seen in at least 8 sessions |
| Poll on the size step, both devices | Open answers coded into themes: not knowing whether to measure the artwork or the frame (mostly phone respondents); "too expensive" (mostly desktop respondents) |
| Usability round, five participants on phones | Three measured the outside of an old frame instead of the artwork. One met the clearing field. All five chose mount and glazing without hesitation |
| Support inbox | Repeated questions asking what frame size a standard print needs; returns noting that the frame was too small for the print |
One funnel, one question, several methods
Every method looks at the same visitors at the same step, so a FunnelModel of the customer journey stages. step chosen wrongly wastes the whole sprint. Choose the step where a plausible improvement would recover the most conversions, not the lowest rate on the report. A finding seen by several methods is evidence; one method gives a lead. When methods disagree, split the segment before concluding.
Task
- Write the sprint question, the single funnel it covers, the owner who decides and the end date.
- Say what the tracking check changes, and which conclusion the unfixed funnel would have led to.
- Calculate the phone and desktop step rates after the fix and choose the step to aim at, with the reason.
- Plan each method at that step: the recording filters and evidence rule, the poll's trigger and question, the usability task written as a goal, the words to search for in the support inbox, and the privacy checks.
- Triangulate the returned findings, and resolve the disagreement in the poll.
- Write the backlog rows with problem, location, evidence, reach, cost as a range, status and decision, and sort them into bugs to fix, hypotheses to write and leads that go back for research.
Worked answer
Question, scope and owner. "Why do phone visitors who start the configurator fail to add a frame to the basket, and what should we change first?" One funnel, from configurator start to basket; the owner decides; the end date is fixed before any data is read.
The tracking check. Before the fix, the phone funnel showed its largest loss at glazing. That loss was a missing event, not missing visitors, and a sprint aimed there would have studied a bug. This is why the check comes first.
Framwick's step rates after the fix
| Step | Phone | Desktop |
|---|---|---|
| Size entered ÷ started | 2,300 ÷ 5,200 ≈ 44% | 2,600 ÷ 3,100 ≈ 84% |
| Mount ÷ size | ≈ 89% | ≈ 90% |
| Glazing ÷ mount | ≈ 95% | ≈ 96% |
| Basket ÷ glazing | ≈ 87% | ≈ 89% |
Phone visitors lost at the size step: 5,200 − 2,300 = 2,900 in four weeks. Matching the desktop rate would mean about 0.84 × 5,200 ≈ 4,370 size entries, a gap of about 2,070. On phones, size entry to basket is 1,700 ÷ 2,300 ≈ 74%, and 55% of baskets become orders. The team's own assumption, stated as an assumption, is that a fix closes between a tenth and a third of the gap: about 207 to 690 more size entries, or 207 × 0.74 × 0.55 ≈ 84 to 690 × 0.74 × 0.55 ≈ 281 orders in four weeks.
The located step. Artwork size on phones: every other phone step runs close to desktop.
The method plan. Recordings: phone visitors who reached the size step and left, current period, evidence rule written before watching, inputs masked, as the recordings and heatmaps tutorial requires. Poll: one open question on the size step when a visitor hesitates, with no personal data in the answer box, read for themes and wording, not shares, as the polls and post-purchase surveys tutorial explains. Usability: a small moderated round on phones with people who frame pictures, test payment details and recorded consent. The task, written as a goal: "You have a print from a trip that you want framed for your hallway. Order a frame for it." It never says "enter the size". Support: search for size, measure, fit and too small, with names removed.
Triangulation. Measuring confusion is seen in recordings, the phone poll answers, the usability sessionsA research session in which a person attempts a realistic task on a site, app or prototype while an observer notes where they struggle. and the support inbox: evidence. The clearing field is seen in recordingsA reconstruction of one visitor's visit, replaying their clicks, scrolling, taps and form interactions, used to see how a page is actually used. and a session, and it is broken: a bug. The poll's price theme comes mostly from desktop respondents, who see the price preview, so split by device: price is not the phone problem, and on desktop it is a single-method lead with no located loss behind it.
Backlog and sort.
- Bug, fixed now without a test: the size field clears when units are switched. Note the fix in the row.
- Hypothesis: visitors do not know whether to measure the artwork or the frame. Location: size step, phones. Evidence: four sources. Reach and cost from the funnel above. Decision: write a hypothesis using the hypothesis tutorial, then check it against the step's traffic with the prioritisation guide.
- Lead, back to research: price on the desktop preview, one method only; a later sprint on the desktop funnel.
The readout follows what, so what, now what: what was found at the size step, what it costs in orders, and what happens next.
Common mistakes
- Aiming at the lowest step rate before the tracking check. The glazing loss was a missing event.
- A usability task that names the field. "Enter your size" tests instructions, not the page.
- Reading the poll as shares of all visitors. Respondents choose themselves.
- Averaging a disagreement away. Split by device first.
- Testing the clearing field. A broken field is fixed and noted, not tested.
The checkpoint that follows in this module tests the same judgements.
