When to run this checklist
Run it after a customer survey is written and before the first invitation goes to the sample, and again before each new wave of a repeated survey or brand tracker. The best time is the day the questionnaire is ready for its pretest, because the questionnaire group needs time for think-aloud sessions and a soft launchReleasing a survey or test to a small share of its audience first, then pausing to read the data for problems before the full launch., and the sample and consent groups need answers from people outside the research team.
The person who ticks it is whoever owns the study, with the analyst who will read the results and whoever owns data protection in the business on the consent and data lines. When the decision to run the survey is part of a wider research programme, the customer research programme playbook decides when a wave is needed and who runs it; this list is the gate for one wave. Interviews, focus groups, usability tests and diary studies have their own gate, the research session readiness checklist, because a moderated session needs recruits, rehearsal and recording rather than a frame and a draw. When the invitation goes by email, the email campaign QA checklist checks the send itself once this list has passed.
Each group belongs to a guide. The research design and brief guide starts from the decision; the pretest tutorial tests the questionnaire; the sampling tutorial draws the sample; the participant information and consent tutorial prepares what people are told and agree to; and the coding tutorial and the research repository guide look after the answers.
Before you start
- The research brief and the analysis plan: the decision, the figures that will inform it, the comparisons the analysis will make and the tables the report will contain.
- The questionnaire built in the tool it will run in, with routing, randomisation and validation switched on and a test link whose answers can be kept apart from real data.
- The change log for the questionnaire: question, problem found, evidence, change made and date.
- The population sentence and the frame, or for a tracker the panel specification and the record of the last wave's draw.
- The business's contact rule for people surveyed recently by any team.
- The name of whoever owns data protection, and the decisions already taken on lawful basis, retention and marketing opt-outs.
- The draft invitation, participant information and incentive terms.
- The name of the person who will code the open answers and of a second person to check the coding.
The checklist
The groups follow the order of the work: the decision first, because every later line is judged against it; then the questionnaire and its pretest; then the sample; then the invitation and the data it creates; and finally the plan for the answers once they arrive. A line that fails in the first three groups stops the launch at that point.
The decision
Questionnaire and pretest
Sample
Invitation, consent and data
Answers and records
Where each check is explained
The widget keeps the lines short so they can be ticked; the reasoning lives in the guides. Each check links to the page that explains it.
| Check | Explained in |
|---|---|
| Decision and decision-maker named | Research design and the research brief |
| Every question maps to the analysis plan | Pretesting a questionnaire |
| Unneeded questions cut, important ones early | Survey and questionnaire design |
| Wording and order read for bias | Survey and questionnaire design |
| Think-aloud sessions with the audience | Pretesting a questionnaire |
| Every route tested on respondents' devices | Pretesting a questionnaire |
| Soft launch read for drop-off and speeding | Pretesting a questionnaire |
| Questionnaire frozen, later changes logged | Setting up a brand tracker |
| Population written down | Drawing a survey sample from a customer list |
| Only the fields the draw needs exported | Drawing a survey sample from a customer list |
| Duplicates, staff, test and no-contact records removed | Drawing a survey sample from a customer list |
| Contact rule applied across teams | Drawing a survey sample from a customer list |
| Draw recorded for the next wave | Drawing a survey sample from a customer list |
| Invitation promotes nothing | Participant information and consent |
| Who, why, how long and what happens to answers | Survey and questionnaire design |
| Lawful basis and retention agreed | Participant information and consent |
| Incentive terms that do not bias who answers | Participant information and consent |
| No personal data in on-page polls | On-page polls and post-purchase surveys |
| Identities separated, deletion date set | Participant information and consent |
| Coder and checker named | Coding open-text survey answers |
| Respondents to be compared with the frame | Drawing a survey sample from a customer list |
| Storage for questionnaire, draw and results | Building a research repository |
How many people to invite is not a line on this list, because the answer comes from the groups the report must read and the business's own past response, not from a rule of thumb. The sampling tutorial and the sample size entry explain the reasoning.
If an item fails
The decision group is the first hard stop. A survey that cannot name the decision it informs, or whose questions do not map to an analysis plan, is not ready for a pretest, let alone a sample; it goes back to the brief, as the research design guide describes. A pretest that keeps turning into a redesign is the same failure found later.
The questionnaire group stops the launch until it passes. Sending the survey is a one-way door in the sense of the one-way and two-way door framework: answers already collected cannot be re-asked, so a broken route or a misread question is fixed, logged and retested before the full send, not noted for the next wave. On a tracker, the freeze line matters most after the BaselineAn estimate of what a metric would have done without a given change, campaign or promotion, against which the effect is measured.: a later change is made only with a logged reason and the wave it took effect in, as the brand tracker tutorial sets out.
In the sample group, records that should not be there are removed before anyone is invited. That includes people inside the business's contact ruleA business-wide rule that a customer surveyed or interviewed recently is not asked to take part again for a set period, whichever team wants to ask., whichever team surveyed or interviewed them last. A frame still holding people on the Suppression ListA list of addresses or contacts excluded from every send, such as unsubscribes, hard bounces and spam complaints., staff or test accounts is fixed now, because those records cannot be taken back out of the results cleanly. An unrecorded draw can go ahead only if the record is written the same day, while the steps are still remembered.
The invitation, consent and data group is the second hard stop. An invitation that promotes a product is marketing whatever its subject line says, as the sampling and consent tutorials note and the comparison of promotional and transactional email shows: remove the offer or do not send. Answers and contact details are Personal DataAny information relating to an identified or identifiable living person, such as a name, email address, location data or an online identifier., so if the Lawful BasisOne of the six grounds the GDPR accepts for processing personal data, such as consent, contract or legitimate interests, chosen for each purpose. and retention have not been agreed with the data protection owner, the survey waits until they have. Incentive terms can be finished on the day, provided the invitation states them before anyone answers.
The answers group can ship with a note and a date. Naming a second coder or a storage place after launch is untidy rather than harmful, but a plan to compare respondents with the frame needs the comparison fields kept in the extract before it is deleted, so check that the sample group kept them.
A language school sends its first student survey (illustrative)
Brezzolt Language School checks a survey of past students
Brezzolt Language School, an invented evening language school, is deciding whether to move from 10-week courses to 6-week ones, and the director wants to hear from past students first. The owner of the study runs this list with the school's part-time analyst and its office manager, who owns data protection.
The decision group passes once the director writes down the decision and that she will make it in March; two of the 14 draft questions, about the coffee machine and the website, map to nothing and are cut. In the questionnaire group, four think-aloud sessions with former students find that one agreement scale runs from "agree" on the left on a laptop but stacks with "disagree" on top on a phone, so two participants answer backwards. The scale is rebuilt to read the same way on both, the route for students who left mid-course is walked again, and a soft launch to 60 people shows no question with unusual drop-off. The questionnaire is then frozen and the change log records three changes.
The sample group finds 2,140 past students in the booking system. Cleaning removes 85 duplicates, 4 former teachers who once took a course, 1 test account the booking system's supplier had created and 230 people who asked not to be contacted. The contact rule removes nobody, because the school has never surveyed students before, and the draw is written up in half a page.
The invitation says who is asking, why, that the survey takes about 7 minutes and that answers are reported only in groups, and it contains no course offer. The office manager agrees the basis and a deletion date for the extract, and the incentive, a book voucher for everyone who takes part, paid also to anyone who stops part-way, is stated in the invitation. Twenty-one lines pass on the day; the coder line ships with a note, because the second coder is named a week later.
Common mistakes
- Writing questions before naming the decision. Every question then looks useful, and the survey grows until nobody finishes it.
- Pretesting with colleagues only. They read each question as intended, so the audience's misreadings never surface.
- Launching to everyone at once. Without a soft launch, a question that drives people away does so across the whole sample.
- Treating the email list as the population. The list is a frame with gaps, and the report has to say who it misses.
- A discount in the invitation. It turns the survey into marketing and changes who answers.
- Consent decided by the survey owner. The basis and retention belong to whoever owns data protection, and a decision nobody wrote down cannot be shown later.
- No plan for the open answers. Free text collected without a coder becomes a pile of quotes chosen to fit a story.
- Changing a tracker mid-series without a log. Waves before and after the change answer different questions.
The checklists library holds the lists that sit beside this one, from email campaign QA to the launch of a test.
