Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Optimisation Event Test

A design for testing which event a campaign should optimise for when purchases are scarce, judged on cost per purchase rather than the optimised event.
Experimentadvanced7 min readUpdated

Hypothesis

If a low-volume campaign optimises for a more frequent upstream event instead of purchase, it will leave the learning phase and deliver purchases at a lower cost, because delivery gets enough signal; if the upstream event predicts purchases poorly, purchases will cost more. The decision behind it is concrete: optimise a low-volume campaign for purchases or for a more frequent upstream event such as add to cart.

Both halves of the hypothesis are live. Meta describes the Learning Phase as the period in which the delivery system is still exploring how best to deliver an ad set, and the learning phase guide explains why a campaign with few purchases may never leave it and lists a more frequent event as one remedy. The remedy has a cost: delivery then finds people who add to cart, and in some shops those people buy at a steady rate while in others many are browsing, comparing or saving for later. Which is true for a given shop is an empirical question, and this test answers it.

Metrics and guardrails

The primary metric is cost per purchase in both arms: spend divided by purchases, with purchases counted the same way for both, ideally reconciled with the shop's own orders. CPA on purchases is the only fair basis, because the upstream arm will always win on its own event. The CPA calculator puts both arms on the same footing. Where the winning event will also be used on other platforms, the guide to conversion events and values across ad platforms records it in a signal register against the result it stands for.

Three guardrails explain the result. Learning phase status shows whether the upstream arm achieved what it was meant to: if it is still learning at the end, the mechanism the hypothesis relies on did not happen. Purchases per optimised event, the share of add-to-carts that became purchases in the upstream arm, shows how well the upstream event predicts buying; a falling ratio means delivery is finding people who add to cart and stop, and the shop's normal Add-to-Cart Rate funnel is a useful reference point. Average order value is the third: an arm that finds cheaper purchases of smaller baskets may not earn more.

Design

Use the platform's A/B test tool with the optimisation event as the single variable. Build two ad sets that are identical in audience, creative, placements, budget, schedule, bid strategy and attribution setting; one optimises for purchase, the other for the chosen upstream event. Both arms track purchases with the same pixel and server events.

Design elementChoiceReason
Unit of randomisationUser, assigned by the platform's A/B test toolKeeps each person in one arm
ArmsOptimise for purchase versus optimise for an upstream eventOne change: the event delivery is told to find
Everything elseIdentical audience, creative, budget and bid strategyIsolates the event from other levers
Primary metricCost per purchase in both armsThe outcome the business pays for
Event trackingSame purchase and upstream events, checked before launchMiscounted events decide the test
Read windowAfter both arms leave or settle in learningLearning-phase cost is not representative

Choose the upstream event by how closely it tracks purchase in the shop's own data. Add to cart is common; initiate checkout sits closer to purchase and fires less often. The campaign objectives guide explains which events each objective allows, and the TikTok campaign structure guide shows where the optimisation goal sits in a TikTok ad group if the test runs there instead. Before launch, confirm that the event used is the same Key Event the business reports in analytics; the GA4 key events and conversions guide covers how those are defined, so the platform and analytics count the same thing.

Sample size reasoning

The test is read on purchases even though one arm optimises for something else, so the scarce event sets the sample. That is the uncomfortable centre of this design: the campaign has too few purchases to optimise for them, and the test needs enough purchases per arm to tell the arms apart. The sample size and significance guide shows how a low event rate stretches the run, and its section on tests that cannot answer the question applies directly.

If purchases are too few to separate the arms within a tolerable period, the honest answer is a longer test or a decision on judgement. A short test that reads noise as a result is worse than either, because it gives a false reason to change the account.

Duration

Run both arms through their learning phase, then at least two full weekly cycles. Extend rather than read early when purchases are sparse. Do not edit either ad set during the test; a significant edit returns that arm to learning and puts the two in different states. If the shop has a long consideration period, allow for purchases that arrive after the read window closes by fixing an attribution window before launch.

Reading the results and the decision rule

Read the test once, at the planned end. Compare cost per purchase between arms with an interval, then check learning status, purchases per optimised event and order value. Write the decision rule into the plan before launch:

ResultDecision
Upstream arm has lower cost per purchase and order value holdsSwitch to the upstream event; recheck when purchase volume grows
Upstream arm leaves learning but purchases cost moreStay on purchase; the upstream event predicts buying poorly here
Upstream arm cheaper but order value fallsStay on purchase, or weigh the margin before switching
No reliable differenceStay on purchase and solve volume by consolidating ad sets
Neither arm leaves learningConsolidate first; the event alone cannot fix the structure

Consolidation is often the better remedy whatever the result, because one ad set with the whole budget gathers purchases faster than several with a share each. The academy capstone on running AI in production for a year at a children's shoe retailer practises applying this to a wide-fit range whose ad sets are merged before any event is tested, within a plan that keeps every platform test and its reading clear of the back-to-school season.

Illustrative example — not a real result

A ceramics shop tests add-to-cart optimisation

Wickerby Ceramics, an invented online pottery shop, records about 40 purchases a month from Meta and its ad set stays in learning limited. It runs an A/B test at £60 a day per arm for six weeks: one arm optimises for purchase, the other for add to cart. The purchase arm records 34 purchases at £74 each and never leaves learning; the add-to-cart arm leaves learning in its second week and records 41 purchases at £61, with order values within a few pounds. The interval on the difference is wide and crosses zero. Wickerby consolidates its three prospecting ad sets into one, keeps purchase optimisation, and plans to retest at the higher volume.

Pitfalls

Judging each arm on its own optimised event is the pitfall that the design exists to prevent, and platform reports invite it, because each ad set's headline result is the event it was given. Set the report's columns to show purchases and cost per purchase for both arms before launch. Too few purchases are the second: a test that starts with a volume problem cannot escape it, so plan for a longer run or a judgement call.

Pixel or server events miscounted are the third. If the add-to-cart event fires twice on some pages, or the purchase event is missing for some payment methods, the arms are judged on bad data. Test every event on the live site before launch and compare platform purchases with the shop's orders weekly during the test.

Common mistakes

Event tests fail on their reading more often than on their setup. The audience targeting breadth test uses the same A/B test structure to compare targeting, and the experiment library lists every design.

  • Reading the upstream arm on add-to-carts. It wins by construction; read purchases.
  • Changing audience or creative with the event. Two changes answer neither question.
  • Reading inside learning. Calibration cost distorts early figures.
  • Ignoring order value. Cheaper purchases of smaller baskets may not pay.
  • Expecting the event to fix a fragmented structure. Consolidate ad sets first.
  • Trusting events without testing them. A double-firing tag decides the result.

Frequently asked questions

Why judge the add-to-cart arm on purchases when it was told to optimise for add to cart?

Because the business pays for purchases, not for baskets. An arm optimising for add to cart will almost always produce cheaper add-to-carts, which says nothing about whether it produces cheaper customers. Both arms must be read on the same outcome or the test compares two different things.

Does this test work on TikTok as well as Meta?

The design does, because both platforms let the ad group or ad set optimise for a chosen event and both describe a learning period after launch. Run it through whichever platform's split testing feature is available, keep the arms identical apart from the event, and read both on purchases from the business's own records.

What if neither arm leaves the learning phase?

Then the event change was not enough on its own, and the volume problem needs a structural answer: fewer ad sets sharing the budget, a broader audience, or a larger budget on one ad set. Record the result, consolidate, and consider retesting only once the structure gives either arm a realistic chance of stable delivery.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Part of these learning paths

Also related

Sources

  1. About the learning phase — Meta Business Help Centre
  2. TikTok Ads Help Center — TikTok for Business

Related