Definition
A usability test is an observed attempt at a task. A participant is given a goal, such as booking an appointment or ordering a sample, rather than a list of instructions, and tries to achieve it while a facilitator watches and a note-taker records what happens. Participants are usually asked to think aloud, so the observers hear the reasoning behind each hesitation. The output is a list of problems, each with where it happened, how serious it was and the participant's own words.
The conversion research methods guide groups research into quantitative, behavioural and attitudinal methods, then sets usability testing apart: it is the one method that reliably finds problems nobody knew to ask about, such as a label that means something different to customers or a step that looks finished when it is not.
In practice: tasks, participants, moderation and note-taking
A session round has a small number of fixed ingredients:
- Tasks written as goals. "You need a new pair of glasses; book whatever you need" leaves the participant to find the route. "Click Book now" tests nothing.
- Participants who resemble real visitors, on the devices and browsers the analytics show they use, not the team's own laptops.
- Moderation, or a script. In moderated sessions a facilitator is present, live or remote, and can ask follow-up questions, which suits complex flows and prototypes. In unmoderated sessions the participant follows a written script and records themselves, which is cheaper and scales to more people but loses the chance to ask why.
- A facilitator who does not help. A rescued participant hides the problem the session existed to find.
- Severity ratings. Each problem is rated by how badly it affected the task, from blocking completion down to minor irritation, and the serious ones are fixed and retested.
The tutorial on running a usability test on a conversion journey adds the preparation that keeps sessions from failing: consent and storage for recordings, test accounts and payment details that let a participant finish without placing a real order, and a pilot session with a colleague. The methods guide's practical rule is to run a handful of sessions, fix what they found and run another handful, rather than many sessions at once.
What usability tests find that analytics and tests cannot
Each method answers a different question. A funnel report says where people leave. A Session RecordingA reconstruction of one visitor's visit, replaying their clicks, scrolling, taps and form interactions, used to see how a page is actually used. shows what they did before leaving. A usability session adds the reasoning, and it works on journeys that have no traffic yet. An A/B TestControlled comparison of two variants. then measures whether a change moved the rate, but only where there is enough traffic to reach a reliable answer. The guide to conversion work without A/B tests uses usability sessions as the check before shipping on low-traffic sites: they say whether people can use a change, not how much it will lift results.
The method has a known bias. People who know they are watched try harder and complain less, so a problem seen in a session is probably worse in real use, while a journey every participant completes is not proven sound. Findings are therefore logged with severity in the backlog kept in the conversion research hub, and a conversion research sprint places a small round of sessions after the analytics and recordings have located the step to test.
An optician's booking flow on phones
Glintmarrow Opticians takes eye test bookings online. Analytics showed that 61% of phone visitors who started the booking flow left at the first step, against 34% on desktop. The team ran five moderated sessions on phones with people who had booked an eye test in the past year, each given the goal of booking a test for next week. Four of the five stopped at the first step to look for the price, and two said aloud that they assumed a fee would appear later. The team added the price of a standard eye test to the first step and reran three sessions, in which nobody hesitated there. Over the next month, completed phone bookings rose from 212 to 268.
Related terms
A usability finding becomes a HypothesisA testable statement of the change to make, the effect expected on a named metric and the evidence-based reason for expecting it. once it names the problem, the change and the metric it should move. A Session RecordingA reconstruction of one visitor's visit, replaying their clicks, scrolling, taps and form interactions, used to see how a page is actually used. shows real visitors' behaviour without asking them, which complements the reasoning a session captures. An A/B TestControlled comparison of two variants. measures the effect of the fix where traffic allows. Sessions are often run on a Landing PageThe page a visitor arrives on from a campaign, built to turn that visit into one action. or the first steps of a journey, because that is where a misread offer stops people before they start.
Common confusions
- Usability tests measure conversion. They find problems and their causes; the size of the effect comes from analytics or a test.
- More participants in one round is better. Small rounds with fixes in between find more serious problems than one large round.
- The team can stand in for customers. People who know the business cannot see the page as a newcomer does.
- A usability test is an interview. Interviews ask about past decisions; a session watches an attempt at a task.
- Every participant finishing means the journey is fine. Observed participants try harder than real visitors, so a clean round is reassuring rather than proof.
