What this checkpoint covers
Yorvanelle Car Care is an invented maker of car shampoos and waxes, sold online and through motor accessory retailers. Its agency has delivered a marketing mix model (MMMStatistical model of channel impact.) recommending a move into online video. The questions apply the tutorial on preparing data for a mix model, the guide to evaluating a marketing mix model and the comparison of mix modelling and incrementality testing, on top of the forecasts and tests of the advanced marketing analytics level.
Yorvanelle Car Care: the agency's model in summary
History: 156 weeks. Fit on the fitted weeks: 0.94; nothing held out. Recommendation: move £150,000 a year from paid search and paid social into online video.
| Channel | Spend pattern | Marginal return per £1 (interval) | Tested |
|---|---|---|---|
| Television | Two bursts a year | £1.40 (£0.60 to £2.20) | Never |
| Online video | On and off with paid social | £2.90 (£0.80 to £5.00) | Never |
| Paid search | Steady | £1.10 (£0.90 to £1.30) | Last year; result used as the prior |
| Paid social | On and off with online video | £0.90 (£0.50 to £1.30) | This spring |
| Retail media | Billed monthly | £1.80 (£0.50 to £3.10) | Never |
Refitted without eight weeks, online video's share of contribution falls from 22% to 9%. When online video spend doubled, the range also went into 140 more stores; store count is not a control. The spring geo test measured paid social at £0.30 per £1 (£0.10 to £0.50); the model puts it at £0.90 (£0.50 to £1.30).
Task
- Write a decision and a reason for each question; a right decision with the wrong reason is a miss.
- Reread the section behind each miss.
Questions
- For the next refresh, the ecommerce manager offers to build the dataset in-house: online orders plus the sales retailers' ad platforms report, missing retail media weeks filled with the channel's average, and the whole history, including the years before the brand stopped selling through a wholesaler. What do you change?
- Online video and paid social have always moved together. What should next year's media plan do about it?
- The appendix says television's prior came from the media plan, every channel shares one carry-over rate and the baseline is flat. What do you challenge, and what do you ask for?
- The agency says the fit proves the channel split. What do you ask for, and what does the refit already tell you?
- The sales director notes that online video's best quarter was also the quarter of the new store listings. What follows?
- The agency calls the model calibrated because it matches last year's search test. The social test comes in below it. How do you read each, and what is tested next?
- Should the recommended move go ahead?
Answers explained
- Outcome, gaps and history. The outcome comes from the system of record across every route, so retailer sell-out replaces platform-reported sales. If the media ran in a missing week, rebuild it from schedules and log the fix; an average hides the gap. Cut the history at the change of route to market rather than stitching across it. See assembling the weekly dataset.
- Planned variation. Alternate their pulses or weight them differently by region, enough to show in weekly data, agreed with both owners and logged in the controls calendar. An IncrementalityConversions caused by marketing. test on one is often the only clean separation. See planning variation.
- All three. An unchallenged prior returns as the answer; one carry-over rate ignores that search usually acts quickly while television may act over weeks; a flat baseline pushes slow growth into media that grew alongside it. Ask for refits under reasonable alternatives and see which estimates survive. See specification choices.
- A holdout and a stability check. Many splits fit the same history; fit only shows whether a driver is missing. A holdout predicts weeks the model never saw. The refit is a failed stability check: online video's share more than halves when a few weeks drop out, so no decision rests on it. See fit, holdout and stability.
- An omitted control. Online video has probably absorbed the distribution gain. Add store count and refit before anything moves. See whether the story makes sense.
- The first agreement is not independent; the new test is the stronger evidence. A result fed in as a prior agrees by construction. The social test measured one channel at one spend against a Holdout GroupA randomly chosen part of an audience kept from exposure to measure what exposure caused., so it becomes a calibration point in the refit; watch where its lost credit lands. Test online video next: widest interval, largest proposed move, never tested. See calibrating the model with tests.
- Not yet. The rule, written before reading results, moves money only where the receiving channel's Marginal ROASExtra revenue returned by the last increment of ad spend. beats the losing channel's at the pessimistic ends of both intervals. Online video's low end sits below both donors' high ends and its estimate failed stability, so refit and test first, then move in steps within the spend the history covers, recorded in the budget reallocation memo template. See uncertainty and the decision.
Where to go next
Misses on review and calibration send you back to the mattress brand exercise. Then start module three with the exercise on valuing leads and sharing costs for a commercial pest control firm.
