Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Google Ads › Advanced

Reading a Geo Split Bidding Test for a Machinery Dealer

You read a finished geo split test in which an invented garden machinery dealer moved half its matched regions to automated bidding. The exercise practises checking the match, setting the learning period aside, reading lift per matched pair and against spend, and letting guardrails rather than the headline decide whether to roll out, extend or stop.
ExerciseAdvanced levelModule 2: Testing bidding and budget changes5 minUpdated

Why this matters

A geo split is an A/B Test run on regions, and it can be read wrongly in ways that look rigorous. The judgement lies in whether regions were comparable, spend stayed level, the right weeks were read and the extra conversions were the right ones. The bidding strategy geo split test sets out the design; this exercise is the read at the end.

The dealer, the regions and the test

Burdockhatch Garden Machinery is an invented dealer of mowers, ride-on tractors and garden tools, selling online and through showrooms with click-and-collect. Orders are counted from its own order system by delivery or collection postcode. The team matched regions into pairs on order history, switched one region in each pair to a target CPA strategy, and kept the other on manual bids.

Illustrative example — not a real result

Burdockhatch Garden Machinery: test set-up and pre-registered rules

  • Pre-period: six weeks before the switch. Learning: two weeks, excluded. Read period: six weeks.
  • Starting target: £30, near the pre-period average. Business maximum cost per order: £40.
  • Contribution margin: 30% of order value.
  • Pre-registered rules: roll out if orders rise at level spend, cost per order stays under £40, and value per order falls by no more than 5% against prediction. Spend in test regions must stay within 5% of its pre-period level relative to control.
Illustrative example — not a real result

Burdockhatch Garden Machinery: orders by matched pair

PairTest preControl preTest readControl readNote
A240250262230Weekly orders tracked closely in the pre-period
B180175196170Tracked closely
C150155168160Tracked closely
D120118128112Tracked closely
E9014085150Weekly orders moved independently; control opened a showroom in the read period
Pairs A to DSpend preSpend readValue per order preValue per order read
Test regions£21,000£24,150£410£360
Control regions£21,500£21,300£405£400

Learning fortnight, pairs A to D: test regions recorded 98 orders against 116 predicted. The store manager wanted to stop the test on that figure.

Reading lift against guardrails

Predict each test region from its control, scaled by their pre-period relationship. Read pair by pair before adding up, and drop any pair whose match failed or whose control was disturbed. If spend did not stay level, the test has partly become a budget test, and extra orders must be judged at their marginal cost, as in the budget increase incrementality test. Then the guardrails: a strategy optimising order count can find smaller orders, which only value per order reveals.

The learning fortnight is not evidence either way. The guide to results dropping after a bid strategy change explains why early weeks look worse and why late days are incomplete until delayed orders arrive.

Task

  1. Decide which pairs belong in the read, and why.
  2. Calculate predicted and actual orders and the lift for the pairs you keep.
  3. Check the spend rule, then recalculate efficiency as orders per pound against prediction.
  4. Estimate the cost of each extra order and compare it with the business maximum.
  5. Check value per order and total conversion value against prediction.
  6. Decide: roll out, extend or stop. Then describe the next test.

Worked answer

Illustrative example — not a real result

Burdockhatch's read, pairs A to D

Predicted test orders = control read × (test pre ÷ control pre): A 230 × 240 ÷ 250 = 220.8; B 170 × 180 ÷ 175 ≈ 174.9; C 160 × 150 ÷ 155 ≈ 154.8; D 112 × 120 ÷ 118 ≈ 113.9. Total predicted ≈ 664.4 against 754 actual: about 90 extra orders, a lift of about 13.5%.

Spend: test regions rose 15% (£21,000 to £24,150) while control fell about 1%, a relative rise of about 16%, outside the 5% rule.

Orders per £1,000: test 32.9 to 31.2; control 32.5 to 31.5. Test efficiency was about 2% below prediction.

Level-spend prediction for test spend: £21,000 × (£21,300 ÷ £21,500) ≈ £20,805. Extra spend ≈ £3,345 for about 90 extra orders: roughly £37 each, under the £40 maximum. Average cost per order in test regions: £24,150 ÷ 754 ≈ £32.

Value per order: predicted £410 × (£400 ÷ £405) ≈ £405; actual £360, about 11% lower, outside the 5% rule. Conversion value: 754 × £360 ≈ £271,400 against 664.4 × £405 ≈ £269,000, about £2,400 more. At a 30% margin that is roughly £700 of contribution for £3,345 of extra spend.

Pairs. Drop E: it never tracked its control, and the showroom opening disturbed the control during the read.

What the lift means. The extra orders are real, but they came from extra spend, not better bidding: at level spend the new strategy was slightly less efficient. The cost per extra order passes the guardrail, yet the orders were smaller, and the value they added does not cover what they cost. The Incrementality that matters here is in contribution, not order count; check the product mix before the next test.

Decision. Stop. Two pre-registered rules failed, and extending would only gather more evidence about a strategy optimising the wrong thing. The learning fortnight played no part in the verdict.

Next test. Rematch or replace pair E, cap test budgets at the pre-period level, and test maximise conversion value with a target return derived from the margin rather than a cost target. Keep the same pre-registered rules, with value per order and contribution after ad cost as the primary readings.

Common mistakes

  • Keeping a broken pair because it is data. A failed match adds noise in a known direction.
  • Reporting the headline lift. It mixes the strategy's effect with the spend that drifted up.
  • Judging the learning fortnight. It was excluded before the test began for good reason.
  • Passing on cost per order alone. Value per order is where count-based strategies fail.

Check this judgement in the bidding and budget tests checkpoint. For how cost and value strategies differ, see the Smart Bidding strategies guide and the intermediate Google Ads level.

Pages this lesson applies

Terms used on this page

Also related

Related