Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Digital Marketing Foundations › Advanced

Checkpoint: Designing Measurement Across the Mix

Scenario questions set in an invented pet insurer test whether you can match a measurement method to a decision, explain what attribution misses, judge when mix modelling is premature, weigh privacy resilience, run an experiment backlog and act when a test contradicts the attribution report.
CheckpointAdvanced levelModule 2: Measurement design across the mix4 minUpdated

What this checkpoint covers

The questions are set at Furzecombe Pet Insurance, an invented insurer selling policies online, with television, online video, paid search and paid social in its mix. They draw on the guide to choosing an attribution approach, the mix modelling versus multi-touch attribution comparison and the experimentation programme playbook.

Task

  • Write a one-line answer and the action you would take for every question before reading on.
  • Mark each answer confident or unsure.
  • Read the explanations; for each miss, write down the assumption that led you astray.
  • Reread the linked section for any confident miss before starting the next module.

Questions

  1. Furzecombe's leadership asks two things in one meeting: should next year's television budget shrink, and which of two social ad messages should run next month? Which method answers each?
  2. The data-driven attribution report gives online video almost no credit, so the paid social manager wants its budget. What is the report unable to see?
  3. A newly launched sister brand, a year old with spend that has hardly changed month to month, wants a mix model. What would you tell it?
  4. Consent rates have fallen and a browser change has shortened cookie life. Which of Furzecombe's methods lose accuracy, and which hold up?
  5. The experiment backlog in the example below has more ideas than the team can run this quarter. Which goes first, and what must every finished test leave behind?
  6. A regional holdout shows paid social adding far fewer policies than the attribution report credits it with. What happens to the budget and to the reports?
  7. Two weeks into a planned four-week landing page test, the channel owner sees the variant ahead and wants to call it. What do you say?
Illustrative example — not a real result

Furzecombe Pet Insurance: experiment backlog

IdeaEvidence behind itReachEffortReadable in a quarter?
Geo holdout on online videoAttribution and platform disagree sharply; £400,000 a year at stakeAll new customersHighYes, on quote starts
Quote form with fewer fieldsSession recordings show drop-off at step twoAll quote startersLowYes
New hero image on the cat pageOpinion from a stakeholderCat page visitors onlyLowBorderline
Price-led versus cover-led social messageSurvey says buyers compare cover firstSocial audiencesMediumYes

Answers explained

  1. Television: mix modelling, checked by experiments; the message: a split test. A budget question across a channel with no clicks needs an aggregate method, which mix modelling provides if the history supports it. The message question is small, fast and reversible, so an A/B Test inside the platform answers it.
  2. Views that never become clicks, and effects that arrive weeks later. User-level Attribution shares credit among tracked touches near the conversion, so upper-funnel video that builds demand later captured by search looks idle. Do not move the budget on the report; test it.
  3. Not yet. A MMM learns from variation over a long history; flat spend over a short period gives it nothing to separate. Start with clean tracking, one reporting model and a first test on the largest line, and build history with deliberate spend variation, as the staged roadmap in the attribution approach guide describes.
  4. Path-based methods weaken; aggregate methods hold. Attribution covers a shrinking, non-random share of journeys. Mix modelling and geo tests use aggregate outcomes and need no identifiers, so they become more important, not less.
  5. The video holdout. It carries the most money, the strongest evidence of a real question and the widest reach; the hero image has no evidence and should go back to intake. Every finished test leaves a learning log entry: question, design, result, confidence, decision, owner and whether the decision shipped. The experimentation programme playbook sets the fields.
  6. The test decides the budget; the reports carry the correction. Reduce social towards the level the test supports, apply a dated incrementality factor to its attributed figures in routine reporting, and record the disagreement and its expiry date. Do not quietly switch attribution models to make the numbers agree.
  7. Let it run. An early lead is not a result; peeking and stopping inflate false winners. Only the analyst declares a result, at the planned end, against the design written before launch.

A learner with six or seven sound answers, all reasoned from the decision rather than the method, is ready to move on. Anything fewer means rereading the guides behind the misses.

Where to go next

If questions two, three or six caught you out, redo the sofa brand measurement exercise. Otherwise start the final module with the outdoor kit growth options exercise, and use the intermediate digital marketing level to refresh the KPI basics these methods report against.

Pages this lesson applies

Terms used on this page

Also related

Related