Start from the decision, not the method
The most common way a study fails is that it was commissioned as a method. Someone wants "a survey" or "some focus groupsA moderated discussion among a small set of people who share an experience, used to hear how people talk about a category together and react to material.", the fieldwork is run, and a report arrives that answers a question nobody asked. The marketing research hub sets out the chain that avoids this: a decision, the questions that would settle it, then a method that can answer those questions in the time and money available.
Working backwards from the decision changes the method more often than teams expect. A decision about which of two propositions to lead with is a measurement problem that needs numbers from a defined population; a decision about why a segment is not buying is a discovery problem that needs conversations; a decision about whether a market is large enough to enter may need no primary research at all, because the evidence already exists in trade and regulator data. The method follows. When the question is still vague, the right first method is usually the cheapest one that sharpens it, which is why most programmes begin with a desk review and a handful of interviews rather than a survey.
Design, qualitative, quantitative, secondary
Four kinds of choice recur in every study. The table names each, the question it settles and the typical options.
| Choice | The question it settles | Typical options |
|---|---|---|
| Design | What the study has to deliver, by when, at what confidence, for whom | Exploratory, descriptive or causal; one-off or tracked over time; in-house or agency |
| Secondary | What is already known, and whether it can be trusted | Published reports, regulator and official statistics, trade bodies, the business's own data, previous studies |
| Qualitative | Why people think and act as they do; what the team does not know to ask | Interviews, groups, observation, diary studies, open-ended feedback |
| Quantitative | How many, how much, how often, and whether a difference is real | Surveys, behavioural data, controlled experiments such as an A/B TestControlled comparison of two variants. |
The choices are not alternatives. A well-designed study usually draws on secondary evidence first, uses qualitative work to find the right questions, and only then measures with a quantitative method. Skipping the middle step is the single biggest cause of surveys that measure the wrong thing precisely.
The core guides
- Research design and the research brief — how to turn a decision into research questions, when to stop at secondary evidence, how to choose a method family, the trade-offs between sampling, timing and budget, and a brief template that keeps the study honest whoever runs it.
- Qualitative vs quantitative research — what each family can and cannot tell you, where each wins, how to sequence them in one programme, and a mixed-methods example from a single study.
The execution of specific methods lives in the customer research section: the interviews tutorial, the survey design guide and the further guides beside them cover fieldwork. Research into why a page or a checkout loses people has its own methods and its own evidence (recordings, heatmaps, on-page polls) and is covered by the conversion research methods guide rather than here.
Further guides
Further guides cover the work around a study: what participants are told before they take part, what a survey promises about their names, buying research from outside, writing up what a study found and keeping findings where the next study can find them:
- Participant information and consent for research — a tutorial for what people are told, how consent is recorded, incentives, withdrawal and deletion, prepared before the first invitation.
- Anonymous, confidential or attributable surveys — choosing the promise a survey makes about names, what each promise allows and rules out, saying it in the invitation and keeping it in the data and the report.
- Commissioning a research agency or freelancer — when outside help pays, the brief, judging proposals, agreeing data handling and checking fieldwork.
- Reporting research findings so they get used — leading with the answer to the decision, showing the evidence and its strength, saying what the study cannot, and recording the decision.
- Building a research repository — a tutorial for what to keep from each study, tagging findings so they can be found, keeping personal data apart and retiring old evidence.
Sampling and bias in brief
A sample is the group of people a study actually hears from, and every finding is a statement about that group first and the wider population second. The questions to ask of any sample are who was eligible, how they were reached, who agreed to take part and who did not. Each stage can distort the result. A survey of newsletter subscribers says little about people who never subscribed; interviews with the customers an account manager volunteered will be the happiest customers; a poll on a website hears only from people who stayed long enough to see it.
Three biases do most of the damage. Selection bias is the gap between the people reached and the people the decision concerns. Non-response biasThe gap between the people a survey or study invited and those who answered; engaged, satisfied customers tend to reply more, so results lean their way. is the gap between the people reached and the people who answered, and it grows as response falls. Measurement bias is the gap between what people say and what is true, driven by leading questionsA survey or interview question whose wording contains or suggests its expected answer, producing measurement bias rather than evidence., social desirability and the limits of memory. None can be eliminated, but all can be reduced and all should be stated in the findings.
How large a sample needs to be depends on what will be done with it. For discovery, the test is whether new participants are still producing new themes. For measurement, it depends on the size of difference that matters and the confidence required, and those calculations belong to the sample size and significance guide in the CRO section. What a marketing team needs to remember is that a small sample read carefully beats a large sample read carelessly, and that a conversion rate (CVRShare of visits or clicks that convert.) or a KPIA metric tied directly to an objective. movement measured on a biased sample is not made true by its precision.
Ethics, consent and data protection
The ESOMAR and MRS codes set out obligations that apply to any organisation gathering information from people for research, whether or not it belongs to either body. Participation is voluntary and based on informed consent; participants are told who is collecting the data and for what purpose; their answers are confidential unless they agree otherwise; research is never a pretext for selling, list-building or collecting debt; and extra care applies to children and vulnerable adults. Findings are reported with their methods and limits, not dressed up as more certain than they are.
Data protection law adds duties wherever the study holds personal data, including recordings, contact details and free-text answers that identify someone. The practical direction is to collect only what the study needs, tell people plainly what will happen to it, separate names and contact details from responses as early as possible, and delete identifying material once the study is closed. Where a study crosses borders or involves sensitive categories, the detail changes and legal advice is needed; a marketing guide can only set the direction. The participant information and consent tutorial prepares the documents and records in order before anyone is invited: what people are told, one consent statement for each use, incentive terms, withdrawal and deletion, with the lawful basis and retention agreed with whoever owns data protection.
A regional bank chooses a method
Ardent Building Society wants to know why applications for its first-time-buyer mortgage stall after the affordability check. The product team proposes a survey to all 4,300 people who started an application in the past quarter. The research lead instead runs a two-week design stage: she reads the 60 most recent call-centre notes on stalled applications, then interviews twelve applicants, six who completed and six who stopped. The interviews show that most people who stopped did not understand what the check had decided and assumed they had been declined. The survey that follows asks about that specific moment, goes to 1,500 applicants and is answered by 410. Nearly 70% of those who stopped say they believed the application had failed. The society rewrites the results page, and the design stage cost less than a tenth of the survey budget the product team had proposed for the wrong questions.
Common research method problems
| Problem | Usual cause | Guide |
|---|---|---|
| The brief names a method but not a question | Study commissioned as a deliverable rather than a decision | Research design and the brief |
| A survey measured things nobody understood | No qualitative stage to find the right questions and vocabulary | Qualitative vs quantitative research |
| Findings only reflect the easy-to-reach customers | Selection and non-response bias unexamined; sample frame chosen for convenience | Research design and the brief, Drawing a survey sample from a customer list |
| Interviews were read as if they were counts | Qualitative evidence treated as a measurement | Qualitative vs quantitative research |
| A test "proved" something on too few conversions | Experiment run without the sample-size work that belongs to the CRO section | Sample size and significance |
| Research into a weak landing page produced opinions, not behaviour | Wrong method family for a conversion question | Conversion research methods |
| The study is being questioned by the legal team | Consent, purpose or retention not planned before fieldwork | Research design and the brief, Participant information and consent |
| Answers promised anonymity are wanted for a follow-up or a link to customer records | The survey's promise about names was never chosen against the decision | Anonymous, confidential or attributable surveys |
| An agency's findings cannot be checked or reused | Slides accepted without bases, question wording or a method statement | Commissioning a research agency |
| The readout ended without a decision | Report written in the order the research was done; recommendations with no owners and no record kept | Reporting research findings |
When a finished study has already been received politely and changed nothing, or its results are disputed as unrepresentative, the marketing research troubleshooting section starts from the decision the study was for and traces the fault to the stage where it entered.
