Hypothesis
Raising channel budget in selected regions will add conversions above what control regions gain, at a marginal cost per acquisition the business can afford. The hypothesis has two parts, and both must hold. The first is that extra spend causes extra conversions at all; for channels that mainly reach people already intending to buy, that is not guaranteed. The second is that the extra conversions come at a cost the business can live with, which requires deciding before the test what "affordable" means.
Set that threshold first. The marginal returns and forecasting guide explains why the cost of the next conversion rises as spend grows, and how to work out the most the business can pay for one from margin and customer value. The CPA calculator helps compare the average cost with the threshold. Write the hypothesis, the threshold and the size of the budget lift into the experiment hypothesis template. Industry bodies such as the IPA have long argued for judging marketing on the effects it causes rather than the credit it receives, which is the spirit of this test.
Metrics and guardrails
The primary metric is marginal cost per incremental conversion: the extra spend in test regions divided by the extra conversions those regions gained over what the control regions predict. It is the true price of growth from this channel at this level of spend. Count conversions from your own sales or lead systems across all channels, not from the platform being tested, because the point is to see whether total conversions rose.
Blended CPACost of each conversion. across all channels is the first guardrail. If the extra spend mainly takes credit from other channels, platform-reported cost may look fine while the blended figure worsens. Branded search volume is the second, and it works in both directions: a rise in test regions suggests the extra spend is creating demand, which supports the result, while a fall in branded paid conversions alongside a rise in the tested channel suggests the spend is capturing demand that would have arrived through search anyway.
Design
Choose regions that can be targeted separately and that each produce a steady flow of conversions. Match them into groups using a pre-period of historical conversions, then assign regions within each group at random to test or control. In test regions, raise the channel's budget by the planned amount; in control regions, change nothing. The incrementality testing guide covers the matching and estimation methods.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Region, within matched groups | Budget cannot be split by person |
| Variants | Current budget versus raised budget | One change: spend level |
| Pre-period | Several weeks or months of history | Establishes the normal test–control relationship |
| Budget lift | Large enough to produce a detectable change | Small lifts vanish in noise |
| Duration | At least one full purchase cycle after spend rises | Captures delayed conversions |
| Measurement | All-channel conversions from your own systems | Detects cannibalisation |
The lift needs to be large enough to see. A small budget rise produces a small number of extra conversions that cannot be told apart from regional noise. The marketing budget calculator helps plan the overall budget that the lift sits within.
Sample size reasoning
The precision of a budget test comes from the number of regions, the length of the read period and the stability of the relationship between test and control regions in the pre-period. A larger budget lift also helps, because it produces a larger expected effect. Few regions with very different behaviour make a weak test; many regions that move together make a strong one.
Use the pre-period to estimate how much the test–control relationship normally wanders from week to week. The effect you hope to see must be clearly larger than that wander. The general logic of effect size and confidence is set out in the sample size and significance guide; geo-specific methods appear in the incrementality guide. If the estimate says the test cannot detect an affordable effect, increase the lift or the duration before starting, not after.
Reading the results
At the end of the read period, estimate the conversions test regions would have achieved without the lift, using control regions and the pre-period relationship. Subtract that from actual test-region conversions to get incremental conversions, and divide the extra spend by it to get the marginal cost. Compare that marginal cost with the affordability threshold set before launch.
If the marginal cost is below the threshold and the guardrails hold, extra budget is justified at this level; the budget allocation guide explains how to fold the result into the wider plan. If it is above the threshold, the channel is already past the point where more spend pays, even if its average cost looks healthy. If incremental conversions are close to zero, the channel is mostly harvesting demand that exists anyway. In every case, record the result where the MMMStatistical model of channel impact. team can use it to calibrate the model. When the channel is Google Ads, the guide to scaling Google Ads spend shows which lever inside the account to pull next and how to read each smaller step at the margin between tests.
A meal kit brand tests a paid social budget lift
Greenplate, an invented meal kit brand, wants to raise paid social spend. It matches twenty regions into ten pairs using six months of order history and raises budget by 50% in one region of each pair for six weeks. Test regions spend £60,000 more than planned. Against the control-based prediction, they gain about 1,200 extra first orders, a marginal cost of roughly £50 per incremental order. The affordability threshold was £45. Branded search rises slightly in test regions. Greenplate raises budget by a smaller amount and plans a second test.
Pitfalls
Auction saturation is the first pitfall. In some channels, extra budget in a region mostly buys the same people more often or bids up the price of the same auctions. The test captures this honestly, but teams sometimes blame the test when the real answer is that the channel is saturated.
Halo across channels is the second. Extra spend in one channel can lift branded search, direct visits and even retail sales, and if only the tested channel's conversions are counted, the test understates the effect. Count all-channel conversions. Seasonality is the third: a lift that coincides with a seasonal peak in some regions but not others will mislead, which is why matching on history and randomising within groups matter. Finally, avoid changing creative, offers or targeting during the test; any change made in only some regions confounds the result.
The size of the lift itself can be planned with the marketing budget calculator for scaling spend, which prices only the extra conversions wanted at the marginal cost of the last budget step.
Common mistakes
Budget tests carry real money and real political weight, so the mistakes below tend to be expensive. If the question is how to bid rather than how much to spend, use the bidding strategy geo split test; the experiment library lists the remaining designs.
- Trusting platform-reported returns. Credit is not causation; the test exists to separate them.
- A budget lift too small to see. The extra conversions disappear in regional noise.
- No affordability threshold. Without one, any positive result is called a success.
- Counting only the tested channel. Halo and cannibalisation both live in other channels.
- Reading before the purchase cycle completes. Late conversions are missed and marginal cost looks worse.
- Changing creative in test regions. That adds a second cause to the comparison.
