Marketing mix modelling and incrementality testing at a glance
A mix model starts from the total. It takes a long weekly history of an outcome such as sales, breaks it into a baseline, the contribution of each channel and the effects of price, promotions and season, and estimates how each channel's contribution responds to its spend. The marketing mix modelling guide covers its inputs and its two core ideas, carry-over and saturation, which together produce a response curve for every channel and show where Diminishing ReturnsEach extra unit of spend or effort buys less extra result than the one before it. set in. Its output is a plan: what the outcome would be under a different split of the same money.
An incrementality test starts from one decision. It changes spend for some people or places and not for others, then compares outcomes. The incrementality testing guide covers the designs, from user-level holdouts to geographic tests, and how to size and read them. Its output is a single causal number: what this channel added at this spend in this period.
Neither replaces the other because they fail in opposite ways. The model sees everything and is certain of nothing in particular; it can be wrong about any channel whose spend moved with another or whose effect was absorbed by a missing control. The test is certain about one thing and blind to everything else; it says nothing about the channel at double the spend, or about the channels it left alone. The effectiveness work the IPA publishes is a common reference point for planners weighing this kind of evidence, and reading it well means knowing which kind of method produced each finding.
Compare table
Marketing mix modelling vs Incrementality testing
| Row label | Marketing mix modelling | Incrementality testing |
|---|---|---|
| Question answered | How does each channel contribute to the total, and what would another split deliver? | What did this channel cause, at this spend, in this period? |
| Data needed | A long, consistent history of spend by channel, outcomes and controls | Randomisable people or regions, outcome data for both groups and a pre-agreed design |
| Channels covered | Every channel in the history at once, including offline media | One channel or one change per test |
| Causal strength | Partial: a model of history, only as good as its controls and specification | Strong within the test's limits: randomisation supplies the counterfactual |
| Time to result | Months to assemble data and build; faster on refresh | Weeks per test, plus planning and a cool-down |
| Cost | Data assembly, modelling expertise and ongoing maintenance | Withheld spend or reach, design effort and analyst time per test |
| Refresh cadence | Periodically, often quarterly or with each planning round | Repeated on the largest lines when spend, creative or market changes |
| Use in budget allocation | Direct: response curves compare the next unit of spend across channels | Indirect: one well-measured point on one channel's curve |
The causal strength and use-in-allocation rows pull in opposite directions, and that is the whole argument for running both. The method best placed to allocate a budget is the one least able to prove cause, and the method that proves cause cannot allocate on its own.
When to choose marketing mix modelling
Choose a mix model when the decision is the split of a whole budget. Annual and quarterly planning asks where the next unit of money should go across every channel, and only a method that estimates all the response curves together can compare them. A test can say one channel is worth keeping; it cannot say whether the money would earn more elsewhere.
Choose it when important media leave no trace a test can easily isolate. Television, radio, out-of-home and sponsorship reach everyone in a market at once, which makes clean holdouts hard and sometimes impossible. A model fitted to a long history of their spend, alongside everything else, can still estimate their contribution.
Choose it when the business has the history. A model needs a long, consistent run of weekly data with real variation in each channel's spend, and the controls that explain the rest of the outcome. A business with a short history, a single channel or spend that never changed gives the model nothing to learn from. It also needs an owner: a model that is built once and never refreshed becomes a historical report.
The tutorial on preparing data for a marketing mix model shows how that weekly history is assembled, checked for flat series and channels that move together, and documented in a data log.
When to choose incrementality testing
Choose a test when one budget line is in question. Whether to keep, cut or grow a large or contested channel is a causal question about that channel, and an experiment answers it more directly than any model. Remarketing, branded search and any channel whose reported return looks too good to question are the usual first candidates.
Choose it when the business is too small or too young for a model. A few digital channels, a short history and a small team can still run one well-designed test a quarter on the largest line, and that produces more decision-ready evidence than a model built on thin data.
Choose it to check a model. Any channel where the model's estimate is uncertain, surprising or about to drive a large reallocation deserves a test before the money moves. That is the bridge to the next section.
A pet food maker calibrates its model with one test
Pondhollow Pet Foods sells through supermarkets and online and spends £1.2 million a year across television, online video, paid social and search. Its first mix model estimates that online video returns £1.60 of revenue for each £1 of spend, but with a range from £0.40 to £2.80, because video spend has always risen and fallen with paid social. Before moving money into video, the company runs an eight-week geographic holdout, withholding video from a matched set of regions. In the treated regions video spend is £80,000 and incremental revenue £96,000, a return of £1.20. The modellers feed that result into the next refit as a calibration point. Video's estimate settles at £1.30 with a range of £0.90 to £1.70, and some of the credit the first model had given video moves to paid social. The planned shift into video is halved, and paid social, now the more uncertain of the two, is scheduled for the next test.
Decision table
| Situation | Choose | Why |
|---|---|---|
| Splitting next year's budget across many channels, offline included | Marketing mix modelling | Only a model compares every channel's next unit of spend at once |
| Deciding whether to keep or cut one large digital line | Incrementality testing | The question is causal and specific to that line |
| Short history, few channels, small team | Incrementality testing | A model would have too little variation to learn from |
| Model's estimate for a channel is wide or surprising | Test that channel, then refit the model | The test turns an uncertain estimate into a calibrated one |
| Television or sponsorship carries a large share of spend | Marketing mix modelling, with geographic tests where feasible | Mass media are hard to hold out cleanly at the level of individuals |
| Finance asks whether marketing's returns are real | Both, with tests as the evidence the model is anchored to | Tests prove cause; the model extends it to the whole budget |
Calibrating the model with tests
Calibration is the work that turns two separate methods into one system. A test gives a measured point on one channel's response curve: at this spend, in this period, the channel added this much. A model calibrated to that point is constrained to agree with it, either by using the result as a prior on the channel's effect or by checking the fitted curve against it and adjusting the specification until it passes through. The other channels' estimates then improve too, because credit the model had wrongly given to the tested channel has to go somewhere.
Choose what to test from the model. The best candidates are channels with the widest uncertainty, the largest spend, the most surprising marginal return or a planned reallocation large enough to matter. A channel whose estimate is tight and whose budget is stable can wait. This keeps testing focused on the lines where an error would cost the most.
Match the test to the model's grain. A model works at channel level by week, so a test is most useful when it measures the same channel definition, on the same outcome, at a spend level the model has seen. A test on one campaign inside a channel, or on an outcome the model does not use, is hard to feed back. Record every test with its dates, spend level, outcome and interval, so each refit can use the full set. The academy checkpoint on mix model data, review and calibration tests how a mix model and an incrementality test calibrate each other and which channel to test first.
Agreement with past tests is one of the checks in the guide to evaluating a marketing mix model, alongside the model's data, specification, stability and uncertainty.
Keep the loop running. Each planning round starts from the latest calibrated model, moves money in steps rather than all at once, tests the channels that moved most, and refits. The guide to choosing an attribution approach places this loop alongside attribution, which handles decisions below the model's grain, and the comparison of marketing mix modelling and multi-touch attribution covers how the model and path-based credit divide the remaining work.
Common mistakes
- Treating a mix model's output as proven cause without testing the channels whose estimates drive the largest moves.
- Generalising one test to the whole channel, at any spend level and in any season.
- Testing a channel definition the model does not use, so the result cannot be fed back into it.
- Moving the whole budget on one refit instead of in steps that can be tested and corrected.
- Building a model and never refreshing it, so a one-off report ends up guiding years of plans.
- Discarding either result when they disagree, instead of asking why and refitting with the test as a calibration point.
- Starting with a model when the business lacks history or variation, when a test on the largest line would have answered the real question sooner.
