Why this matters
Most measurement arguments are about which number is right. The better question is which number is fit for which decision. A high-value product with a long consideration period breaks the easy answers: there are too few orders for quick tests, journeys span weeks and devices, and the channel under most suspicion, online video, rarely earns a click. The guide to choosing an attribution approach sets the rule that the money and irreversibility of a decision set the standard of evidence; this exercise asks you to apply it when the ideal instrument is out of reach.
The brand and its decisions
Frithwood Sofa Company is an invented direct-to-consumer sofa maker selling online. Buyers usually order fabric swatches, return to the site several times and order weeks later. The team is a marketing lead and a freelance media buyer, with no analyst.
Frithwood Sofa Company: channels, volume and data
| Item | Detail |
|---|---|
| Annual media | £540,000: online video £180,000, paid search £200,000, paid social £160,000 |
| Orders | About 150 a month, average order value about £1,400 |
| Swatch requests | About 2,500 a month; roughly one in six swatch requesters orders within ten weeks |
| Consideration | Typically four to eight weeks from first visit to order |
| History | 20 months of weekly spend and sales; online video began 8 months ago |
| Tracking | Analytics with data-driven reporting available; about 60% of visitors accept analytics cookies |
| What reports say about video | Last click credits it with 3% of orders, data-driven 9%, the video platform claims 25% |
Decisions leadership makes each quarter: the split of budget across the three channels; whether online video stays, grows or goes; which search campaigns and social audiences get more of their channel's money; which creative message leads (comfort or design); and the return figure the board sees.
Matching methods to decisions
The mix modelling versus multi-touch attribution comparison frames the choice: mix modelling (MMMStatistical model of channel impact.) works top down from aggregate spend and sales and suits channel budgets, while user-level AttributionAssigning conversion credit to touchpoints. works bottom up from tracked journeys and suits optimisation inside channels. Experiments, which measure IncrementalityConversions caused by marketing. directly, keep both honest. For each decision, ask three things: what instrument answers it in principle, what data and volume that instrument needs, and whether this team can own it. When the answer to the last two is no, choose the best instrument the brand can run now and put the ideal one on the roadmap.
Task
- List the decisions and rate each on money at stake and how hard it is to reverse.
- Choose the method that should answer each decision, and say why the alternatives fail.
- State the data, outcome and volume each chosen method needs, and whether Frithwood has it.
- Pick the outcome each test should measure, given how few orders arrive.
- Write a staged roadmap: what to do this quarter, next, and once the data allows.
- Name the governance rules that stop the methods producing three competing answers.
Worked answer
Channel split. In principle a mix model, because it covers video without clicks. In practice Frithwood has under two years of history, video has run for only part of it and spend has barely varied, so a model would have little to learn from and nobody to refresh it. For now, split on marginal evidence from budget steps and tests; the model waits.
Video: stay, grow or go. The biggest contested line, expensive to get wrong, and invisible to path-based reports. Run a geo holdout: pause video in matched regions and compare with regions that keep it. Orders are too few to read in a reasonable time, so make swatch requests the primary outcome and orders a secondary read over a longer window.
Within-channel choices. Platform attribution for bids and audiences, and analytics data-driven attribution for reporting, with last click kept as a reconciliation baseline. Cheap, fast and good enough, because these decisions are small and reversible.
Creative message. A platform split test on paid social, judged on swatch requests, logged and read by someone other than the person who proposed it.
Board return. Revenue from the order system as the record, with the analytics split as supporting detail and the video test result applied as a dated adjustment.
Frithwood: roadmap and volume check
| Stage | When | Work |
|---|---|---|
| Foundations | This quarter | Swatch request defined as a key event; consistent campaign tagging; a reconciliation table of platform versus analytics, published monthly |
| First test | Next quarter | Video geo holdout across 10 matched region pairs for 8 weeks plus a 6-week order read; swatch requests primary |
| Programme | Following two quarters | One test a quarter on the largest open question, next on paid social; learning log kept |
| Model | Once about three years of weekly data exist with real spend variation | A mix model built with outside help and calibrated against the test results |
Volume check: about 2,500 swatch requests a month gives a test enough events to read in weeks; about 150 orders a month does not.
Governance follows the experimentation programme playbook and the measurement plan tutorial: one system of record per outcome, every figure labelled with its source and model, test results dated with an expiry, and a learning log owned by the marketing lead. When the video test and the analytics split disagree, the test decides the budget and the disagreement is recorded.
Common mistakes
- Commissioning a mix model first. Too little history and variation; it would answer with false precision.
- Judging video on last click. The method cannot see the effect the channel is bought for.
- Testing on orders alone. Too few to read; use a frequent upstream outcome and confirm on orders later.
- Choosing what the team cannot own. A sophisticated method nobody can explain loses to a simple one trusted.
- Unlabelled numbers. Three methods are fine; three unlabelled answers are an argument.
Test your judgement in the measurement design checkpoint, or return to the intermediate digital marketing level for the KPI tree that these methods report against.
