Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Audience Targeting Breadth Test

A test plan for broad or Advantage+ audiences against detailed interest targeting on Meta: a platform split test, cost per purchase and new-customer guardrails.
Experimentintermediate6 min readUpdated

Hypothesis

If an ad set uses broad or Advantage+ audience targeting instead of detailed interests, with the same creative, budget and optimisation event, it will deliver purchases at a lower cost, because the delivery system finds buyers outside the interests the team would have chosen. The decision behind it is concrete: hand audience selection to the platform, or keep detailed interest targeting.

The hypothesis has a known failure mode, and the design is built around it. A broad audience includes the business's existing customers and recent site visitors, and the delivery system tends to find people likely to buy, who are often those same people. Broad targeting can then show a lower cost per purchase while buying mostly repeat orders, which looks like a win and is not. The Advantage+ audiences guide explains how the option expands beyond its suggestions and when manual targeting still wins; this page turns that question into a test plan for one account and reads it on new customers as well as cost.

Metrics and guardrails

The primary metric is cost per purchase, checked against the business's own order data. Read purchases from Ads Manager to compare the arms, then reconcile each arm's purchase count with orders in the shop's records for the same period, so a difference in tracking or attribution does not decide the test. The CPA calculator keeps the arithmetic consistent across arms; CPA here is spend per purchase, not per click or per add to cart.

Three guardrails decide whether a lower cost is a real gain. New-customer share of purchases, matched against the customer file, is the most important: if broad targeting wins on cost but its purchases come mostly from existing customers, the decision rule keeps interests. Frequency is the second: a broad audience is larger, so frequency should be lower; if it is higher, delivery is concentrating on a small group, often recent visitors. Average order value is the third, because cheaper purchases of smaller baskets can leave revenue flat.

Design

Use Meta's A/B test tool with targeting as the single variable. Build two ad sets in the same campaign structure: one with the account's best detailed interest targeting, the other with broad or Advantage+ audience. Creative, placements, optimisation event, budget, schedule and attribution setting are identical. Apply the same exclusions to both arms, normally a Custom Audience of existing customers and recent purchasers, using the methods in the audience exclusions and overlap guide.

Design elementChoiceReason
Unit of randomisationUser, assigned by the A/B test toolKeeps each person in one arm, so audiences do not overlap
ArmsDetailed interests versus broad or Advantage+ audienceOne change: targeting breadth
Creative, budget, optimisation eventIdentical, frozen for the testIsolates targeting from the other levers
ExclusionsSame customer exclusions in both armsMakes the test about prospecting, not retention
Primary metricCost per purchase, reconciled with own ordersJudges on the outcome the business pays for
LearningBoth arms through the learning phase before the readDifferent learning states distort early cost

If the account already uses a Lookalike Audience as its prospecting audience, it can be the control instead of interests; the decision is still whether to hand selection to the platform. Choose the control the account would really run, and keep the comparison to two arms unless the account has the volume to feed three. The academy exercise on planning a model railway retailer's tests of platform automation practises this design with the interests the account already runs as control and the customer exclusion matched in both arms, then schedules it ahead of an optimisation event test on the same platform so that test keeps whichever targeting wins.

Sample size reasoning

The arms are compared on purchases, so the test needs enough purchases per arm to separate the smallest cost difference that would change how the account is built. A small difference is not worth rebuilding the prospecting structure for, so set the threshold first. The sample size and significance guide sets out how the purchase rate and that difference fix the run length, and why cost per purchase needs more data than a click metric.

If the account cannot produce enough purchases per arm within a tolerable period, either test on the largest prospecting campaign, accept that only a large difference can be read, or decide on judgement and record that no test supported the choice.

Duration

Run both ad sets through the learning phase, then at least two full weekly cycles of stable delivery. Meta describes the learning phase as the period after launch or a significant edit when delivery is still being calibrated and performance is less stable, so read neither arm inside it. Do not edit either ad set during the test; a significant edit returns that arm to learning and puts the arms in different states.

Reading the results and the decision rule

Read the test once, at the planned end. Compare cost per purchase between arms with an interval rather than the point estimate, reconcile purchases with the shop's orders, then check new-customer share, frequency and order value. Write the decision rule into the plan before launch:

ResultDecision
Broad wins on cost per purchase, new-customer share holdsMove prospecting to the broad setup; keep the customer exclusion in place
Broad wins on cost but new-customer share fallsKeep interests; broad is mostly reaching existing customers
Interests win on costKeep interests; revisit when creative or signal quality changes
No reliable differenceChoose the setup that is cheaper to manage, usually broad, and watch new-customer share

If broad delivery keeps finding existing customers despite exclusions, the troubleshooting guide for Advantage+ audience reaching customers covers the causes and fixes before any retest.

Illustrative example — not a real result

A skincare brand tests broad targeting against its interest set

Orrinvale Skincare, an invented online skincare brand, prospects with an interest set built around natural beauty and wellbeing. It runs an A/B test with an Advantage+ audience arm, excluding customers from both, at £150 a day per arm for four weeks after learning. The interest arm records 190 purchases at £31.60; the broad arm records 236 at £25.40. Matched against the customer file, new-customer share is 81% in the interest arm and 78% in the broad arm, and average order value is within a pound. Frequency is lower in the broad arm. Orrinvale moves prospecting to the broad setup and keeps the exclusion.

Pitfalls

Existing customers inflating the broad arm is the pitfall that most often produces the wrong decision. The customer exclusion helps, but lists go stale and match rates are imperfect, so the new-customer guardrail must be read from the business's records, not assumed. Different learning states are the second: if one arm leaves learning a week after the other, its early cost drags down its average, so read only the period after both have stabilised.

Creative changed mid-test is the third. A new ad added to one arm, or a winning ad paused in one, makes creative the variable. Freeze creative for the test, and test creative separately with the creative testing guide once targeting is settled.

Common mistakes

Targeting tests fail more often on their setup than their arithmetic. The ad creative hook test varies the creative with targeting fixed and is the mirror of this design, and the experiment library lists every design.

  • Running two ad sets outside the A/B test tool. The audiences overlap and the comparison is unfair.
  • Leaving customers in the broad arm only. Exclusions must be identical in both arms.
  • Judging on platform purchases alone. Reconcile with the shop's orders.
  • Testing broad against a weak interest set. Compare with the targeting you would really run.
  • Editing during the test. A significant edit resets learning in one arm.
  • Ignoring new-customer share. Cheap repeat orders are not prospecting success.

Frequently asked questions

Why must the test use Meta's A/B test tool rather than two ad sets running side by side?

Two ad sets running side by side can reach the same people, because a broad audience contains most interest audiences. The A/B test tool assigns each person to one arm, so the comparison is between targeting approaches rather than between two ad sets competing for the same users.

Should existing customers be excluded from both arms?

If the campaign's job is prospecting, yes, in both arms and in the same way, using a customer list as a custom audience exclusion. The new-customer guardrail then checks whether the exclusion held. If exclusions are left out, the broad arm often finds existing customers first, and its cheaper purchases are repeat orders the business would have won anyway.

Which interest set should the control use?

The one the account actually runs today, or the one the team would choose if broad targeting were not available. Testing broad delivery against a deliberately weak interest set proves nothing; the decision is whether broad beats the best targeting the team would otherwise use.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Part of these learning paths

Also related

Sources

  1. Meta Business Help Centre — Meta
  2. About the learning phase — Meta Business Help Centre

Related