What hand-coding is for
The survey design guide gives the method in one sentence: code open-text answers into themes, using a codeframe built from a sample and then applied to the rest. This tutorial is that sentence done by hand, as part of the survey's measurement rather than as a skim for interesting comments.
Hand-coding earns its time in three situations. When the volume is small enough to read every answer, which the voice of the customer guide treats as the case for coding by hand. When the codes need judgement, such as telling a worry about price from a worry about value. And when the frame will be reused in later waves, because a frame built carefully by hand is the foundation any later machine coding is checked against. For larger volumes, the AI research synthesis guide builds the same kind of codeframe for a model to apply, with checks on every quote and count.
Coding sits between qualitative and quantitative work. The qualitative versus quantitative comparison notes that the closed questions of a survey can be counted while the open answers have to be read and coded, not tallied. Coding makes them countable without pretending they are something else: a count of respondents who volunteered a theme, never a measure of how many customers hold the view.
Other pages code customer words for other jobs. The tutorial on customer language for copy codes jobs, objections and phrases across reviews and tickets to rewrite a page, and the tutorial on on-page polls reads one-question polls for themes. For support teams, the tutorial on tagging support contacts by reason turns the same codeframe discipline into a short reason list that agents apply at the close of every contact. Interview transcripts are synthesised as the customer interviews tutorial describes.
Before you start
Have the questionnaire, the analysis plan and the export of answers in front of you. The export should hold one row per respondent with a respondent ID, the open answer, and the closed answers and SegmentationDividing a market or customer list into groups that need different offers or messages. fields the analysis will cut by, such as customer type, plan or sign-up period. If no plan exists yet, the survey analysis plan template names the coded questions, the coders and the groups before the export is made. With an analysis planA survey's plan, written before the questionnaire: the decision, the figures and comparisons the report needs, which question feeds each, and small-group rules. in place, each open question arrives at coding already tied to the decision it serves.
Decide who will code. Hand-coding is best done by people who know the business but did not write the questionnaire's hopes into it, and it needs a second person for the agreement check. Agree with them how differences will be settled.
Set up a coding sheet: one row per respondent, columns for the answer, the codes applied, a flag for answers that need discussion, and notes. Keep the codeframe on a separate tab where every change is dated.
Check what respondents were told about how their answers would be used. The professional codes the site's research guides describe from ESOMAR and the MRS keep answers confidential unless people agree otherwise, and that promise decides who may see the raw answers and how quotes may be used later.
The steps
Reread the question and the decision it serves
Read the open question exactly as respondents saw it, where it sat in the questionnaire and who was routed to it, and then the decision the analysis plan says it feeds. A question asked of people who rated the service poorly will be read differently from the same question asked of everyone. The decision shapes the frame: codes should separate the things the decision could change, and lump together the things it could not. Write the decision at the top of the codeframe tab, so every code can be tested against it.
Remove personal details from the answers
Free text often contains names, email addresses, phone numbers, order or account numbers and details of other people, such as a member of staff who helped. These are Personal DataAny information relating to an identified or identifiable living person, such as a name, email address, location data or an online identifier. under the GDPR, as the personal data glossary entry explains, and that includes free text that identifies someone when combined with other details. Replace each with a neutral marker, such as "[name]" or "[order number]", in the copy the coders work from, keep the original export where only the survey owner can reach it, and read a sample of the cleaned answers by eye, because identifying details turn up in unexpected places. The coders need the meaning, not the identity.
Read a sample and draft the codes
Read a random sample of answers from across the groups the analysis will compare, without coding them, and write down the themes as they appear. Then draft a code for each, named in plain words close to what respondents said. Resist importing codes from the questionnaire's hopes or the team's assumptions; the frame describes what people wrote. For purchase and switching questions, the jobs-to-be-done framework offers a starting structure that the AI synthesis guide uses: the trigger that started the search, the alternatives considered, the anxieties that slowed the decision and the outcome the customer wanted. Use it as a check on what the draft may have missed, not as a frame to force answers into.
Write a definition, an exclusion and an example for each code
Give every code a one-line definition of what it covers, a line on what it excludes, and an example answer taken from the sample. The exclusion does most of the work: overlapping codes, such as a worry about the price and a worry about whether the product is worth the price, separate cleanly only once each says what belongs to the other. Add the codes that every frame needs: "other", for answers that fit no code, and "no answer or not relevant", for blanks, "don't know" and answers to a different question. Keep the frame short enough that a coder can hold it in mind.
Group codes into nets
A net is a group of related codes reported together, such as several codes about cost under one cost net. Nets let the report give the broad picture and the detail beneath it. Define each net by the codes it contains, and decide now that a net counts each respondent once, however many of its codes they received. Keep nets few and distinct, and keep codes that matter to the decision visible on their own rather than buried in a net.
Code every answer, allowing more than one code
Work through every answer, applying every code that fits; an answer that gives two reasons gets two codes. Code what the respondent wrote, not what you think they meant. Flag answers that are ambiguous rather than guessing, and settle them together at the end of each session. Watch the "other" pile: when several answers in it say the same thing, the frame is missing a code. Add it with a definition, date the change, and recode the answers already coded that might belong to it. A frame that keeps changing late means the sample read when drafting the codes was too small or too narrow.
Have a second person code a portion and settle the differences
Give a second coder the codeframe and a random portion of the answers, without the first coder's codes. Compare the two sets answer by answer. Where they agree, move on; where they differ, discuss each case and work out why. Most differences trace back to a definition or exclusion that allows two readings, and the fix is to sharpen it and recode the affected answers, not to declare a winner. Record the differences found and the changes made. This tutorial uses discussion rather than an agreement statistic; the point is a frame two people apply the same way.
Count respondents per code against the base
Count the number of respondents who received each code and each net, not the number of mentions. State the base beside every count: the number of respondents who were asked the question and the number who gave a codable answer, and say which one the shares use. Because answers can carry several codes, the shares add up to more than the whole, and the report should say so. Cut the counts by the groups in the analysis plan, such as plan, customer type or the CohortA group of customers who share a starting event and period, tracked together over time. who signed up in a given period, but only where the base allows: a share of a handful of respondents is read as the guide to reading small numbers advises, with the count shown and the figure treated as rough. The Sample SizeThe number of users or sessions an experiment needs to detect a given effect reliably. behind a closed question does not transfer to an open one that fewer people answered.
Choose quotes that represent each code
For each code that will be reported, choose quotes that show what the code typically sounds like, not the most vivid or the most quotable. Include a quote that sits at the edge of the code, so readers see its range, and quote dissenting answers where they exist. Copy quotes exactly, with only the personal details already removed. Attribute them by group, such as "annual plan, joined this year", never by name, as the interviews tutorial's checklist asks, and check that no combination of group and wording points to an individual. Freeze the codeframe with its definitions and the date, so the next wave can apply the same frame and compare.
A codeframe on one page
A codeframe fits on one page when it is working well. The table shows its columns, filled for a generic question about what nearly stopped someone signing up. The codes are examples of the format, not a frame to copy.
| Net | Code | Definition | Excludes | Example answer |
|---|---|---|---|---|
| Cost | Price level | The amount seemed high or above budget | Doubts about whether it was worth it (see Value) | "It was more than we'd planned to spend" |
| Cost | Value | Unsure the product would be worth what it cost | The amount itself (see Price level) | "Wasn't sure we'd use enough of it" |
| Effort | Set-up worry | Concern about the time or work needed to start | Moving existing data (see Moving data) | "Looked like a lot to set up" |
| Effort | Moving data | Concern about transferring records from a previous system | General set-up effort | "Didn't want to lose our old records" |
| Trust | Unknown company | Had not heard of the business or doubted it would last | Doubts about a feature | "Never heard of you before" |
| Not a reason | Nothing | Says nothing nearly stopped them | Blank answers (see No answer) | "Nothing really" |
| Not a reason | Other | A codable reason that fits no code | Answers to a different question | Varies; reviewed for new codes |
| Not a reason | No answer | Blank, "don't know" or not about the question | Any codable reason | "n/a" |
The structure is the one the survey design and AI synthesis guides describe: a frame built from a sample, each code with a definition, an exclusion and an example. Every row earns its place by helping a decision; a code nobody would act on can usually be merged. When the counts reach a report, the guide to reporting research findings covers how to present them with their evidence and limits.
A payroll software codes what nearly stopped sign-ups (illustrative)
Taskivor Payroll codes what nearly stopped people signing up
Taskivor Payroll, an invented payroll software for small employers, adds the open question "What, if anything, nearly stopped you signing up?" to its survey of new customers. 640 customers complete the survey and 412 write an answer to the open question.
Before coding, the survey owner replaces employee names, emails and payroll reference numbers in 37 answers with markers. Two coders read a random 80 answers and draft 12 codes in 4 nets. Their first frame has "Price" and "Not sure it's worth it" as separate codes with no exclusions, and in the second coder's check of 60 answers they disagree on 14, mostly between those two. Adding exclusions, so that "Price" covers the amount and "Not sure it's worth it" covers doubts about value, settles most of the differences, and the affected answers are recoded. The "other" pile grows to 19 answers, 11 of which mention moving payroll history from the previous provider, so "Moving payroll history" becomes a code.
Of the 412 respondents who answered, 118 (29%) mention worry about moving payroll history, 96 (23%) the price level, 61 (15%) value, and 88 (21%) say nothing nearly stopped them; across all the codes the shares add up to more than 100% because answers can carry several codes. The report states the base of 412 and the 640 who completed the survey, notes that the respondents are customers who did sign up, and quotes answers attributed by plan and company size. The frame is frozen and dated for the next quarter's survey.
Verify it worked
- Every row in the coding sheet carries a code. Answers that fit nothing carry "other" or "no answer", never a blank.
- Each code has a definition, an exclusion and an example answer on the codeframe tab.
- The second coder matches the first on the shared portion, or each difference has a recorded reason and a sharper definition.
- Each count shows respondents, not mentions, with its base. In the Taskivor example, that is 412 answers out of 640 who completed the survey.
- The shares across codes add up to more than the whole, and the report says why.
If it goes wrong
- Coders disagree on many answers. Two codes probably overlap. Add exclusions, then recode the affected answers.
- The "other" pile keeps growing. The frame lacks a code, or the opening sample was too narrow. Add a dated code, recode earlier answers and read a wider sample.
- The shares add up to exactly the whole. Each answer probably received one code. Recode so every fitting code is applied, then recount.
- A share looks large but few people answered. State the base beside the count and treat the figure as rough.
- A quote could point to a person. Personal details may remain, or the group label is too narrow. Choose another answer.
Next steps
- Read the AI research synthesis guide when volume outgrows hand-coding, and check its machine coding against your frozen frame.
- Use the survey design guide to write a closed question when the business needs to know how common a view is.
- Read the customer interviews tutorial to synthesise transcripts, then set those themes beside your coded counts.
Common mistakes
- Coding before reading a sample. A frame drafted from the first few answers, or from the team's assumptions, misses what respondents actually wrote.
- Codes without exclusions. Overlapping codes are applied differently by different coders, and the counts stop meaning anything.
- Leaving personal details in. Names, emails and order numbers travel from the coding sheet into slides and quotes.
- One code per answer. Forcing a single code loses the second reason and distorts the counts.
- Counting mentions instead of respondents. One talkative respondent can make a theme look common.
- No base, or the wrong one. A share without the number who answered the question cannot be read.
- Reporting themes as customer opinion. Volunteered themes describe what respondents chose to write, not what all customers think.
- Choosing the most vivid quotes. A dramatic quote shown alone misrepresents a code most people described mildly.
- Changing the frame between waves. A recoded frame breaks comparison; add codes with a date rather than redefining old ones silently.
