Scope: what this audit covers and what it does not
The question this audit answers is whether the decisions that rest on the business's research can trust it: whether each study is tied to a decision, sampled and run soundly with consent, analysed and reported honestly, kept where it can be found and acted on, and which weakness should be fixed first. It covers the customer and market research the business runs or buys within the evidence window, from briefs to the repository, agency work, market sizes and competitor monitoring.
It does not re-teach the work, which sits in the marketing research methods section and its tutorials. The continuing rhythm is the customer research programme playbook; a single survey or session is checked before it starts with the survey launch checklist or the research session readiness checklist; one failed study is diagnosed in the marketing research troubleshooting collection.
Evidence from data rather than research, such as attribution and tests of cause, is scored by the marketing measurement audit. Usability research on pages belongs to the conversion research guides, survey scores read as service measures to the customer retention audit, the support operation to the customer service audit and controlled tests to the experiment library. Legal compliance is out of scope. Every scorecard sits in the audit library.
Before you score: access and evidence
Ask for the folder or tool where studies are kept, the recruitment records and the repository, and ask whoever owns data protection to show the consent records, which sit in a restricted log; without them the consent rows stay provisional. Fix an evidence windowThe fixed period of data an audit scores, agreed before scoring, long enough to cover a full cycle for the business and kept the same at every rescore. that includes at least one study whose findings fed a decision, and keep it for every rescore.
Collect:
- the briefs, questionnaires, discussion guides and screeners of the studies run or bought in the evidence window;
- the sample frames, draw records and recruitment logs, with who was invited, who took part and who was excluded;
- participant information, consent records, incentive terms and the deletion date set for each study;
- codeframes, analysis plans and any AI tools or prompts used to summarise transcripts or open answers;
- the reports, readout notes and the decisions recorded after each study;
- the research repository or wherever findings are kept, and the agency proposals and deliverables accepted;
- the current market size workings and the competitor monitoring log.
Before reading any study, list the decisions in the window that cited research. The audit tests whether the evidence each one leaned on could bear its weight.
Criteria and scoring
Rate each row missing (absent or broken), partial (present but incomplete, inconsistent or out of date) or solid (matches the "what good looks like" column). The scale is your judgement against that column, not a number. Score a row across the studies in the window: a practice followed in some and not others is partial. Rows marked gate decide the band on their own when they score missing. A row that does not apply is marked not applicable rather than missing.
Decisions and design
| Criterion | What good looks like | Score guidance |
|---|---|---|
| A decision behind every study (gate) | Each brief names the decision, who takes it, by when and what finding would change it, before a method is chosen | Missing: studies start from a topic / Partial: no owner, date or deciding finding / Solid: all named first |
| Existing evidence searched first | Internal data and published sources are searched and logged with a judgement of each source before primary fieldwork is bought | Missing: fieldwork bought first / Partial: searched, not logged / Solid: logged and judged |
| Method fitted to the question | Exploratory work finds the questions worth measuring and surveys count them, in that order unless the brief says why not | Missing: surveys before any exploration / Partial: method by habit / Solid: chosen, order explained |
The gate follows the guide to research design and the research brief, recorded in the research brief template. Desk researchResearch from existing sources, the business's own and published ones, done first so that interviews and surveys cover only what nobody has answered. is judged against the tutorial on running a desk research pass and method against the comparison of qualitative and quantitative research. The hierarchy of evidence helps match a design to the size of its decision.
Samples and recruitment
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Sample frame and draw recorded | The population is written down, the frame cleaned of duplicates, staff and people who must not be contacted, drawn at random within strata and recorded so the next wave can repeat it | Missing: whoever is easiest to reach / Partial: draw or exclusions unrecorded / Solid: written, cleaned, repeatable |
| Neutral screener and quotas | Participants are described by what they recently did, screened without giving the qualifying answer away, held to quotas for each group compared and replaced only from the same group | Missing: volunteers, or the answer given away / Partial: no quotas / Solid: neutral, held to quotas |
| Respondents compared with the frame | After fieldwork respondents are compared with the frame, gaps are reported, and weighted and unweighted figures are both shown | Missing: never compared / Partial: gaps or weighting hidden / Solid: compared and reported |
| Every buying role heard | Where a business buys, each role in the buying committee and lost prospects are heard, recruited without account owners picking friendly contacts | Missing: the usual contact only / Partial: picked by account owners / Solid: every role, recruited independently |
Frames follow the tutorial on drawing a survey sample from a customer list and each ScreenerThe short questionnaire that confirms each research candidate fits a study, asking about recent behaviour without revealing which answers qualify. the tutorial on recruiting participants and writing a screener. The comparison is the prevention routine in the diagnosis for survey results that are not representative, and buying roles follow the guide to researching business buyers and buying committees.
Instruments and fieldwork
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Questionnaires pretested | Every question traces to the analysis plan, the build is read on respondents' devices, think-aloud sessions and a soft launch find problems and the questionnaire is frozen before full fieldwork | Missing: no pretest / Partial: internal read only, or edited after launch / Solid: pretested and frozen |
| Neutral wording and order | Questions ask one thing each without suggesting the answer, scales are balanced, order effects are considered and stated intent is not read as behaviour | Missing: questions suggest answers / Partial: unbalanced scales or order / Solid: neutral throughout |
| Interviews about past behaviour | Discussion guides reconstruct specific recent episodes rather than ask for opinions or predictions, and interviewers probe without leading | Missing: "what would you buy?" / Partial: episodes planned, opinions asked / Solid: episodes, neutral probes |
| Groups and diaries used where they fit | Groups are chosen only where interaction between people is the point and moderated against dominant voices, and diaries only where behaviour must be caught as it happens | Missing: groups as cheap interviews / Partial: unmoderated / Solid: chosen and moderated |
Pretesting follows the tutorial on pretesting a questionnaire, ending in a Soft LaunchReleasing a survey or test to a small share of its audience first, then pausing to read the data for problems before the full launch., and wording the survey and questionnaire design guide, which cites Nielsen Norman Group's articles and treats each Leading QuestionA survey or interview question whose wording contains or suggests its expected answer, producing measurement bias rather than evidence. as the writer's bias. Interviews follow customer interviews, step by step, whose Discussion GuideThe plan for a research interview or group: a warm-up, the episode to reconstruct, open questions about past behaviour and probes, used as a plan, not a script. is a plan, not a script; groups the guide to running a focus group; diaries the tutorial on running a diary study. Stated and revealed preference explains why stated intent is not behaviour.
Consent and participant data
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Consent recorded for each use (gate) | Participants are told every use of what they say, agree to each optional use separately, and the consent record sits in a restricted log linked by participant code | Missing: no record, or one bundled agreement / Partial: clips, quotes or re-contact not covered / Solid: each use, recorded, restricted |
| Research kept apart from selling | Invitations carry no offer, incentives neither bias nor bind, and answers are never used for sales follow-up without separate consent | Missing: invitations sell, or answers reach sales / Partial: policy only / Solid: kept apart in practice |
The gate follows the tutorial on participant information and consent, which rests on the MRS and ESOMAR codes as the research guides summarise them and applies the GDPR by agreeing each use and how long material is kept with whoever owns data protection: general direction, never legal advice. The second row follows the research methods section, where research is never a pretext for selling; the sampling and recruitment tutorials, in line with the ICO's direct marketing guidance, treat an invitation that promotes a product as marketing.
Analysis and synthesis
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Open answers coded against a codeframe | Open answers are coded with a defined codeframe built from a sample, applied consistently, checked by a second coder and counted against a base | Missing: skimmed for quotes / Partial: no definitions, second coder or base / Solid: defined and checked |
| AI summaries traced to sources | Any AI summary of transcripts or open answers is traced back to the material before it becomes a finding, and personal data is removed before material reaches a tool | Missing: pasted into reports / Partial: sometimes traced, or identifiable input / Solid: traced, de-identified |
| Simulated respondents kept out of findings | Synthetic personas and simulated answers prepare real research at most and are never reported as what customers said | Missing: reported as customers / Partial: mixed with real findings / Solid: preparation only |
Each CodeframeThe defined set of codes open-text answers are tagged with, built from a sample, applied consistently, checked by a second coder and counted against a base. is judged against the tutorial on coding open-text survey answers, with the base fixed in the survey analysis plan template. AI use follows the guide to using AI for research synthesis, a use the consent tutorial says participants must be told about, and simulated answers the guide to synthetic personas and simulated respondents.
Reporting and use
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Answer-first reports with stated limits | Reports lead with the answer to the decision, show the evidence and its strength, use quotes fairly and say what the study cannot tell | Missing: written in research order / Partial: limits or strength hidden / Solid: answer, evidence, limits |
| Decisions recorded after readouts | Each readout ends with recommendations that have owners, and the decision taken is recorded and followed up | Missing: no decision / Partial: no owners or record / Solid: recorded, followed up |
| Findings kept in a repository | Findings are kept as tagged records with their study, date and strength, apart from personal data, and evidence that has aged is retired | Missing: slide decks only / Partial: untagged, never retired / Solid: tagged, separate, retired |
Reports follow the guide to reporting research findings, which notes that the MRS and ESOMAR codes, as the methods section describes them, expect findings reported with their methods and limits. The decision row is the prevention routine in the diagnosis for research findings not acted on, where a Decision LogA running record, one row per decision, of what changed and why, what was expected, who decided and when the result will be checked. shows which findings changed which decisions, and the repository follows the guide to building a research repository.
Bought and market research
| Criterion | What good looks like | Score guidance |
|---|---|---|
| Agency work checked | Agencies are briefed on the decision, judged on the question rather than the presentation, agree data handling and have materials and fieldwork signed off before deliverables are accepted | Missing: accepted as delivered / Partial: data handling or materials unagreed / Solid: briefed, signed off, agreed |
| Market sizes shown as ranges | Each layer of a market size is defined so someone else could apply it, two independent routes are compared and the result is shown as a range with its driving assumption | Missing: one figure from one report / Partial: one route, or undefined layers / Solid: a range with its assumption |
| Competitor monitoring routine | Competitors' positioning, pricing and messaging changes are tracked from sources within ethical limits on a set rhythm and reach the people who use them | Missing: when someone remembers / Partial: never routed / Solid: a rhythm, ethical, routed |
Bought research follows the guide to commissioning a research agency or freelancer, which treats a supplier handling participants' data as a processor under the GDPR. Sizes follow market sizing: TAM, SAM and SOM and the market sizing worksheet template. Monitoring follows the competitive intelligence guide, which, pointing to the Competition and Markets Authority's guidance, rules out exchanging sensitive information with competitors.
How to score and prioritise
Score the two gates first. If studies start without a named decision, mark the quality rows provisional: they describe research nobody could use. If consent was not recorded for each use, stop any use participants did not agree to: the consent tutorial treats every use not listed before fieldwork as closed to the study afterwards.
Then rank the remaining weaknesses in this order:
- Whatever makes findings describe the wrong people: unrecorded frames, recruitment of whoever answers, screeners that give the answer away and respondents never compared with the frame.
- Whatever distorts the answers: untested questionnaires, leading wording, interviews about opinions and AI summaries nobody traced to the material.
- Whatever stops findings being used: reports in research order, hidden limits, no recorded decision and no repository; agency, sizing and competitor practice ranked by the decisions they feed.
Within each group, a flaw in the study behind a large decision outranks the same flaw in a study nobody cited.
Reading the result: what to do with each band
Red means a gate scored missing. Hold new fieldwork until every brief names its decision and every study records consent for each use, and check findings already in circulation against what participants agreed to. Rescore on the same evidence window.
Amber means the gates hold at least partially but several other rows are partial or missing, the usual outcome. Work a short ordered list, one area at a time, and judge each fix on the next study.
Green means the gates are solid and most rows are too. Move effort into the rhythm the customer research programme playbook runs, and rescore each quarter or after each major study.
From findings to a prioritised fix list
Write each partial or missing row as a finding in plain words: what you saw, in which study, and which decision it weakened. Then build one fix list with a line per fix: the finding, the fix, the owner, the guide that explains it and the rescore date, ordered by the priorities above rather than by ease.
Group the lines by who does the work: briefs and readouts with decision owners, frames, screeners and questionnaires with whoever runs studies, consent with that person and the data protection owner, the repository with its keeper and agency terms with whoever signs the order. Show the band, the gate findings and the first fixes to whoever commissions research, and keep the full scorecard for the rescore.
Illustrative example
A pushchair maker audits its research before a range decision
Yuskeprand Prams, an invented pushchair maker, audits the 4 studies it ran or bought in the 12 months before deciding on a new range: a survey, 9 parent interviews, an agency study and a market size. The survey went to 3,000 newsletter readers and 540 answered; nobody compared them with the 18,000 customers on its full list, so the frame and comparison rows score missing. The interview guide asked parents which pushchair they would buy next, so that row scores missing too. Consent covered taking part but not the clips from 6 interviews shown at a staff meeting, so the consent gate scores partial, and the briefs named a topic, not the range decision, so the decision gate scores missing. The agency report spent its first 14 slides on method. Of the 22 rows, 1 is not applicable (it sells only to families), 8 score missing, 9 partial and 4 solid. The band is red. The fix list puts consent for each use and a brief naming the range decision first, a fresh draw from the full list compared with its respondents second, an interview guide built on recent purchases third and an answer-first report from the agency fourth. After the next study both gates score solid, 5 rows remain missing and the band moves to amber.
Common mistakes
- Scoring the questionnaire before the brief. A flawless survey that serves no decision is still wasted.
- Treating engaged customers as the market. Newsletter readers answer first.
- Accepting consent to take part as consent to everything. Clips, quotes and re-contact each need agreement.
- Trusting an AI summary or a simulated respondent as evidence. Trace every finding to real participants.
- Judging bought research by its slides. Its quality sits in the sample, screener and pretest.
- Counting studies instead of decisions. Many studies and no recorded decisions score worse than few acted on.
