Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Glossary term

Usability Test

A research session in which a person attempts a realistic task on a site, app or prototype while an observer notes where they struggle.
AnalyticsprocessUpdated

Definition

A usability test is an observed attempt at a task. A participant is given a goal, such as booking an appointment or ordering a sample, rather than a list of instructions, and tries to achieve it while a facilitator watches and a note-taker records what happens. Participants are usually asked to think aloud, so the observers hear the reasoning behind each hesitation. The output is a list of problems, each with where it happened, how serious it was and the participant's own words.

The conversion research methods guide groups research into quantitative, behavioural and attitudinal methods, then sets usability testing apart: it is the one method that reliably finds problems nobody knew to ask about, such as a label that means something different to customers or a step that looks finished when it is not.

In practice: tasks, participants, moderation and note-taking

A session round has a small number of fixed ingredients:

  • Tasks written as goals. "You need a new pair of glasses; book whatever you need" leaves the participant to find the route. "Click Book now" tests nothing.
  • Participants who resemble real visitors, on the devices and browsers the analytics show they use, not the team's own laptops.
  • Moderation, or a script. In moderated sessions a facilitator is present, live or remote, and can ask follow-up questions, which suits complex flows and prototypes. In unmoderated sessions the participant follows a written script and records themselves, which is cheaper and scales to more people but loses the chance to ask why.
  • A facilitator who does not help. A rescued participant hides the problem the session existed to find.
  • Severity ratings. Each problem is rated by how badly it affected the task, from blocking completion down to minor irritation, and the serious ones are fixed and retested.

The tutorial on running a usability test on a conversion journey adds the preparation that keeps sessions from failing: consent and storage for recordings, test accounts and payment details that let a participant finish without placing a real order, and a pilot session with a colleague. The methods guide's practical rule is to run a handful of sessions, fix what they found and run another handful, rather than many sessions at once.

What usability tests find that analytics and tests cannot

Each method answers a different question. A funnel report says where people leave. A Session Recording shows what they did before leaving. A usability session adds the reasoning, and it works on journeys that have no traffic yet. An A/B Test then measures whether a change moved the rate, but only where there is enough traffic to reach a reliable answer. The guide to conversion work without A/B tests uses usability sessions as the check before shipping on low-traffic sites: they say whether people can use a change, not how much it will lift results.

The method has a known bias. People who know they are watched try harder and complain less, so a problem seen in a session is probably worse in real use, while a journey every participant completes is not proven sound. Findings are therefore logged with severity in the backlog kept in the conversion research hub, and a conversion research sprint places a small round of sessions after the analytics and recordings have located the step to test.

Illustrative example — not a real result

An optician's booking flow on phones

Glintmarrow Opticians takes eye test bookings online. Analytics showed that 61% of phone visitors who started the booking flow left at the first step, against 34% on desktop. The team ran five moderated sessions on phones with people who had booked an eye test in the past year, each given the goal of booking a test for next week. Four of the five stopped at the first step to look for the price, and two said aloud that they assumed a fee would appear later. The team added the price of a standard eye test to the first step and reran three sessions, in which nobody hesitated there. Over the next month, completed phone bookings rose from 212 to 268.

A usability finding becomes a Hypothesis once it names the problem, the change and the metric it should move. A Session Recording shows real visitors' behaviour without asking them, which complements the reasoning a session captures. An A/B Test measures the effect of the fix where traffic allows. Sessions are often run on a Landing Page or the first steps of a journey, because that is where a misread offer stops people before they start.

Common confusions

  • Usability tests measure conversion. They find problems and their causes; the size of the effect comes from analytics or a test.
  • More participants in one round is better. Small rounds with fixes in between find more serious problems than one large round.
  • The team can stand in for customers. People who know the business cannot see the page as a newcomer does.
  • A usability test is an interview. Interviews ask about past decisions; a session watches an attempt at a task.
  • Every participant finishing means the journey is fine. Observed participants try harder than real visitors, so a clean round is reassuring rather than proof.

More terms to look up

Where this term is used

Related