Why research comes before testing
A test is expensive. It takes development time, it occupies a page for weeks, and it consumes traffic that could have answered a different question. A test built on a guess spends all of that for roughly a coin toss. Research changes the odds. When the problem is located in a specific step of the funnel (FunnelModel of the customer journey stages.), on a specific device, among a specific kind of visitor, and when the cause has been heard from visitors in their own words and seen in recordings, the proposed change addresses something real. It can still lose, because the fix may be wrong even when the diagnosis is right, but it loses for an informative reason.
Research also reveals what should not be tested. Broken forms, missing prices, error messages that make no sense and pages that fail on one browser are bugs, not hypotheses, and the correct response is to fix them and move on. A good share of the improvement in a first programme comes from this category and costs nothing in testing capacity. The CRO hub describes the whole loop from research through testing to learning; this section covers the first half of it. The second half, experimentation, lives in the experimentation hub, and the division matters: research decides what to test, testing decides whether it worked, and confusing the two produces tests that cannot fail and research that never ends.
Methods, hypotheses, prioritisation: the core guides
The core guides come in the order the work happens.
Conversion research methods is the catalogue. It groups the methods into quantitative (analytics and funnel reports, which show where people leave and how much each leak costs), behavioural (recordings and heatmaps, which show what they did before leaving), attitudinal (surveys, polls and interviews, which are the only route to why) and usability testing (watching people attempt the task). For each it states what the method can answer, what it costs in effort and what bias it carries, and it closes with triangulation: how to combine methods so that each covers another's blind spot.
Writing test hypotheses is a step-by-step tutorial for the document that connects a finding to a test. It walks from the evidence to a problem statement, names the audience and the page, proposes a change and predicts its effect, chooses a primary metric and guardrails, and records the whole thing in a form that can be reviewed and later judged against the result.
Prioritising tests takes the backlog of hypotheses that research produces and orders it. It covers scoring models and their limits, explains why traffic and the smallest effect worth detecting are the hard constraint most scoring ignores, and shows how to sequence tests that would interfere with one another.
Read them in that order the first time; afterwards each stands alone. The quantitative half of the methods guide has a practical companion in the funnel analysis tutorial, which builds the funnel step by step from the events a site already records.
Further guides
Further tutorials put the methods into practice, from a whole sprint to a single method, and turn what visitors say into the words on the page:
- Running a conversion research sprint — scoping one funnel and one question, checking the tracking, locating the leak, then watching, asking and testing until a backlog is ready.
- Reviewing session recordings and heatmaps — one question, one step and one segment, a tally of behaviours rather than impressions, maps read by segment, and privacy checks before collecting.
- On-page polls and post-purchase surveys — one open question at the right moment, answers coded into themes, and those themes recorded as evidence in the backlog.
- Running a usability test on a conversion journey — testing a checkout, form or sign-up journey: tasks written as goals, recruiting, moderating without helping, notes, severity and fixes.
- Expert review of a conversion journey — an expert or heuristic review with criteria drawn from the guides, every step walked, severity and confidence rated, and the evidence that would verify each issue named.
- Turning customer language into page copy — gathering words from reviews, support, sales calls and polls, coding objections and phrases, then rewriting the page and testing it.
The research backlog
Research produces more findings than any team can act on, and the useful habit is to keep them in one place with the same fields, so that the strongest ones rise and the rest are not lost. A backlog row records the problem, where it was observed, which methods saw it, how many visitors it affects, what it probably costs, and what has been decided. Entries that several methods agree on, that sit at a large step of the funnel and that affect many visitors go to the top.
| Field | What it holds | Why it matters |
|---|---|---|
| Problem | One sentence describing what visitors fail to do, in plain language | Keeps the entry about the visitor, not the proposed fix |
| Location | Page, step and device where it was observed | Locates the test and reveals patterns across entries |
| Evidence | The methods that observed it and what each showed | Entries seen by several methods are more trustworthy |
| Reach | How many visitors pass through the affected step in a period | Decides whether a test could ever detect a change |
| Cost | The orders or leads probably lost, expressed as a range | Ranks problems by what they are worth fixing |
| Status | New, hypothesis written, prioritised, testing, fixed, parked | Stops entries being re-researched |
| Decision | Fix without test, write a hypothesis, or park, with the reason | Makes the backlog a record of thinking, not just a list |
A software company's backlog after one sprint
A business software company runs a three-week research sprint on its trial sign-up flow, which 4,200 visitors enter each month and 610 complete. Funnel analysis puts the largest loss at the company-details step, where 1,900 visitors arrive and 820 continue. Recordings of 40 sessions show repeated attempts to skip the "number of employees" field, and an exit poll on the step collects 115 responses, a third of which mention not knowing why the information is needed. The backlog entry records all three sources, a reach of 1,900 visitors a month, an estimated cost of 150 to 250 lost trials a month, and the decision to write a hypothesis rather than simply remove the field, because sales uses it for routing. Two smaller entries, a broken postcode lookup on one browser and a pricing page that quotes an old price, are marked fix-without-test and closed within the week.
Research cadence
Research is not a one-off project at the start of a programme; visitors, products, traffic sources and competitors change, and findings go stale. A workable rhythm has three layers. Continuous collection runs all the time at low cost: an on-page poll on the key steps, exit surveys, and the funnel report checked weekly for a step whose rate has shifted. A research sprint of a few weeks runs once a quarter or when a funnel is being rebuilt, and goes deeper: recordings reviewed against a question, interviews with recent customers and recent abandoners, and a round of usability sessions. A measurement audit runs whenever the tracking changes and at least once a year, because research built on events that double-fire or that stopped recording is worse than no research.
Each layer feeds the backlog, and the backlog review, monthly in most teams, is where findings become hypotheses or are parked. The research key performance indicator (KPIA metric tied directly to an objective.) is not the number of findings; it is the share of tests whose hypothesis came from evidence rather than opinion, and the share of those that moved the metric in the predicted direction.
Common research problems
Most research goes wrong in a handful of recognisable ways. The table maps each to the guide that addresses it.
| Problem | What it looks like | Guide |
|---|---|---|
| Findings with no location | "Visitors do not trust us" with no page, step or segment attached | Research methods, quantitative section |
| Only one method used | A backlog built from analytics alone, or from a survey alone | Research methods, triangulation |
| Findings that are really solutions | "Add a trust badge" recorded as a problem | Writing hypotheses, problem statement step |
| Hypotheses without a metric | A change proposed with no agreed way to judge it | Writing hypotheses, metric and guardrails step |
| Backlog ordered by opinion | The loudest stakeholder's idea tested first | Prioritising tests |
| Tests the traffic cannot support | Small changes on low-traffic pages that run for months | Prioritising tests, traffic constraint |
| Research that never becomes a test | A long report, no backlog, no decisions | Experimentation hub |
| Research that sprawls | No agreed question, funnel or owner; several funnels studied at once | Conversion research sprint |
| Recordings watched without a question | Hours of sessions viewed and impressions remembered rather than counted | Session recordings and heatmaps |
| Poll answers reported as shares of all visitors | Self-selected respondents read as if they were a sample of everyone | On-page polls and post-purchase surveys |
| Usability sessions that find little wrong | Tasks written as instructions; a facilitator who steps in to help; colleagues as participants | Usability testing |
| A review's opinions sent straight to testing | Issues from one walk-through treated as findings, with no evidence to verify them | Expert review |
| Page copy written in the company's words | Headlines and benefits drafted from the brief rather than from what customers say | Customer language for copy |
The quickest check on any backlog is to read the problem column aloud: every entry should describe something a visitor could not do, name where it happened, and point to at least two sources. Entries that fail that reading go back to the methods guide before they go anywhere near a test.
