Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Glossary term

Incrementality

Conversions caused by marketing.
AnalyticsconceptUpdated

Definition

The incremental effect of a marketing activity is the outcome, such as conversions or revenue, that occurred because of the activity and would not have occurred in its absence. The counterfactual, what would have happened anyway, cannot be observed for the same people at the same time, so it is estimated from a group that did not receive the activity but is otherwise comparable. The registry's one-line summary captures the method: incrementality is measured by holdout or geo experiments rather than by attribution alone. Lift is the usual expression of the result, the outcome in the exposed group relative to the control group, and incremental cost per outcome is the spend divided by the incremental outcomes only, not by the attributed total.

How it works

The design is a randomised experiment, the same logic as an A/B test (A/B Test) with marketing itself as the variable. In a user-level holdout, a random share of the eligible audience is withheld from the ads, often by showing them an unrelated public service ad so that exposure is comparable, and conversions are compared between the exposed and withheld groups. In a geo experiment, markets rather than people are the unit: some regions receive the activity and matched regions do not, and the outcome is compared across regions over the test period, which works when user-level randomisation is impossible, as with offline channels, or when the platform's own holdout cannot be trusted. Either way the result is a lift estimate with an uncertainty range, and the business question is whether the incremental outcomes justify the spend. The incrementality testing guide covers test designs, choosing cells, size and duration, and reading lift and uncertainty.

ApproachWhat it measuresWhat it needsMain limitation
Platform attributionCredit for observed conversions among observed touchpointsTracking and an attribution windowCannot tell caused from coincident
User-level holdoutLift among a randomly withheld share of the audiencePlatform support or own randomisation, enough volumeOnly the audience the platform can reach; cross-device gaps
Geo experimentLift across treated versus control regionsRegions that can be matched, enough regions, a long enough periodSpillover between regions; small effects need large tests
Marketing mix modelContribution of each channel from aggregated dataLong history of spend and outcomesCorrelational unless calibrated against experiments

In practice

Incrementality tests are run on the channels and campaigns whose attributed numbers are most likely to overstate their effect: brand search, remarketing and any activity aimed at people already close to buying. A test that shows low lift on a high-ROAS campaign is the common and uncomfortable finding, and it is the reason the guide to choosing an attribution approach recommends pairing attribution for day-to-day optimisation with experiments for budget decisions. The results are also used to calibrate marketing mix models (MMM), so that the model's channel estimates are anchored to measured lift rather than to correlation alone. The attribution hub gathers the full set of approaches.

Illustrative example — not a real result

A remarketing holdout

A retailer's remarketing campaign reports 2,000 attributed purchases a month on $40,000 of spend, a cost per attributed purchase of $20. A holdout is set up in which 10 percent of the eligible audience sees public service ads instead. Over six weeks the exposed group converts at 4.4 percent and the withheld group at 4.0 percent. The lift is 0.4 points, or 10 percent relative, with a confidence interval that excludes zero. Scaled to the full audience, the campaign caused roughly 200 of its 2,000 attributed purchases, so the incremental cost per purchase is about $200 rather than $20. The campaign still pays, because the margin on an order exceeds that, but the budget is trimmed and the saving is tested on prospecting, where the attributed numbers are worse and the incremental ones may be better.

What it is not

Incrementality is not attribution (Attribution) with a better model: every attribution model, from last click to data-driven, divides observed conversions among observed touchpoints and cannot distinguish a conversion that was caused from one that was merely preceded. It is not a lift study in the sense of a brand survey, which measures awareness or intent rather than outcomes, although the experimental design is similar. It is not something a dashboard reports continuously; it is measured in discrete tests and the result decays as audiences, creative and competition change. And a positive lift is not automatically a good investment; the incremental cost per outcome has to be compared with what an outcome is worth before the spend is called effective.

Attribution (Attribution) answers "which touchpoint gets the credit"; incrementality answers "did the marketing cause it", and a measurement plan needs both because they serve different decisions. Marketing mix modelling (MMM) estimates channel contributions from aggregated data and is strongest when calibrated with incrementality results. The A/B test (A/B Test) is the general experimental method of which a holdout is one application, with the control group receiving no marketing rather than a different version of it. Related designs such as conversion lift, geo lift, synthetic control and matched-market testing are specific implementations covered in the testing guide linked above, and the choice between them is mostly a question of what can be randomised and how much volume is available.

Common mistakes

  • Reading platform-attributed conversions as incremental, especially on remarketing and brand search, where the audience was already on its way to the purchase.
  • Running a holdout too small or too short to detect the lift that would change the decision, and then reading the inconclusive result as proof either way.
  • Letting the control group be contaminated by other campaigns that reach it, so the comparison is between two exposed groups and the lift is understated.
  • Testing once and assuming the lift is permanent as audiences, creative and competition change around it.
  • Treating a positive lift as proof of value without comparing incremental cost with margin, when a real but tiny effect can still be a poor use of budget.

More terms to look up

Where this term is used

Related