Definition
An analysis plan is the written statement of what a survey must deliver, settled before the questionnaire exists. It names the decision the survey informs and the people that decision is about, then lists each figure and comparison the finished report will need, worded as the report sentence it will become with a blank where the number will go. Beside every one of those outputs it records the question, or the field already held on the customer record, that will supply it.
The order is what makes it useful. Once the outputs are written first, drafting questions becomes a test: a question that supplies no output is removed before it uses up respondents' attention, and an output with nothing behind it reveals a gap while it can still be closed. The survey analysis plan template gives the plan as a table to copy and explains each field; this entry describes the idea rather than the form.
What the plan holds, and how it differs from a research brief and a test plan
Beyond the decision and its outputs, the plan settles the choices that are easiest to bend once results are on the screen: the groups answers will be cut by and where each group comes from, how people are drawn and whether answers will be weighted, who codes the open questions, how a group too small to read will be shown, the form the readout will take, and how answers are kept apart from identities. Fixing these in advance is what lets a reader accept that a comparison or a weighting choice was not picked for the result it produced.
Two neighbouring documents are easy to mix up with it. A research brief commissions the study as a whole, from the decision and what is already known to the method, budget and timetable. The analysis plan follows the brief and copies the decision and population across exactly. A test plan is the design of an experiment, with its hypothesis, primary metric and decision rule: it plans how a test will be judged, not what a survey must yield.
In practice: pretesting, coding, cleaning and commissioning
The plan stays open while the later research steps run. The tutorial on pretesting a questionnaire starts by marking the output each question feeds, removing whatever feeds none and checking that every answer format can produce what it is meant to; a pretest that grows into a redesign has exposed a fault in the plan itself.
The tutorial on coding open-text answers reads each open question beside the decision the plan says it serves, so its codes separate the things that decision could change, and it splits the coded counts only by the groups the plan lists. The tutorial for cleaning survey responses and building their tables applies the plan's completion and quality rules before any result is read. Where the plan has none, the rules are written and dated before the file is opened, and the report admits they came after fieldwork.
When an outside supplier runs the study, the plan becomes something to ask about. The guide to commissioning research from an agency or freelancer asks each supplier what its analysis plan will contain, and reads the answer as part of judging how a proposal handles quality and analysis.
A homeware shop finds the missing question before fieldwork
Doltoar Homeware, an invented online homeware shop, offers 3 delivery options and wants to decide whether to drop the slowest. Before any question was drafted, the marketing manager wrote the plan's outputs as report sentences, 5 in all, such as "Customers who last chose standard delivery rate their delivery … points lower than those who chose next-day." Mapping the first draft's 16 questions to those outputs removed 4 that fed nothing. It also showed that neither a question nor an attached field said which option each customer had last used, so 2 of the 5 outputs had nothing behind them. The order records held that detail, so it was attached to each of the 1,200 invitations instead of being asked, and the pretest began with a complete map to check against.
Related terms
A Test PlanThe written design of a test, fixed before launch: hypothesis, unit of randomisation, primary metric, guardrails, planned sample, duration and decision rule. fixes an experiment's design before launch, as the analysis plan fixes a survey's outputs. The CodeframeThe defined set of codes open-text answers are tagged with, built from a sample, applied consistently, checked by a second coder and counted against a base. is built for the open questions the plan names, and its counts are split by the plan's groups. A Soft LaunchReleasing a survey or test to a small share of its audience first, then pausing to read the data for problems before the full launch. opens the survey to part of the sample before the rest, which is when the pretesting tutorial confirms that the incoming data has the form the plan expects. The plan's choices on who is invited and whether to weight are where Non-Response BiasThe gap between the people a survey or study invited and those who answered; engaged, satisfied customers tend to reply more, so results lean their way., the gap between those invited and those who reply, is planned for. In a repeated survey, each Survey WaveOne round of fieldwork in a repeated survey or tracker, drawn, worded and coded the same way as earlier rounds so the results can be compared over time. keeps the earlier wording unchanged wherever the plan reads a trend.
Common confusions
- A topic is an output. "Views on delivery" cannot be matched to a question. A sentence with a blank for its figure can, which is why the plan words outputs that way.
- Every possible comparison belongs in it. The plan names the comparisons the decision needs. Listing every cut encourages a hunt for any difference and leaves groups too small to read.
- It can never change. It can be revised during pretesting, before the questionnaire is frozen. A comparison added once the data is in is reported as exploratory.
- Every study needs one. It is written for a survey that will inform a decision. A quick on-page poll that is read for themes, not shares, can do without one, and an experiment has its own test plan.
