Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Existing Customer Exclusion Test

A geo experiment for whether prospecting ads should exclude existing customers: matched regions, cost per new customer, repeat orders as the guardrail.
Experimentadvanced8 min readUpdated

Hypothesis

If prospecting campaigns exclude existing customers in a set of matched regions, the cost per new customer there will fall while orders from existing customers across all channels hold, because those customers would have bought through other routes, unless the prospecting ads were the route that won their repeat orders. The decision behind it: exclude existing customers from prospecting campaigns, or leave them in and let the campaigns reach them.

The guide to excluding existing customers sets the exclusions and the customer acquisition goal, reads the new-customer share before and after, and lists when not to exclude, including a business whose margin comes from repeat purchase and which has no other way to win it. A before-and-after reading shows whether new-customer share rose. It cannot show whether repeat orders were lost, because those orders fall in channels the campaign never sees. That is the gap this design fills.

Metrics and guardrails

The primary metric is cost per new customer in each region, counted from the business's own customer records: prospecting spend in the region divided by the new customers the region produced across all channels in the window. Define a new customer once, before the test, using the guide to new and returning customers in marketing data, which sets out how to record that status at the moment of conversion and how long a customer counts as existing. Do not use the platforms' own new-customer labels; each sees only its own lists. The CPA calculator turns each region's spend and new customers into a cost per new customer to compare with CAC elsewhere.

Three guardrails, each with a limit written before launch:

  • Orders and margin from existing customers across all channels, by region. This is the guardrail that decides. Count every order from existing customers in the region, through email, search, direct visits, marketplaces and any other channel the business records, and the margin on them. The repeat purchase and loyalty guide explains why many stores depend on those orders.
  • Total new customers acquired, by region. A cheaper cost per new customer means little if the treated regions bring fewer new customers in total because the platforms lost delivery.
  • Share of the customer list the platforms matched. An exclusion only removes the customers a platform can recognise. Record the matched share per platform at the start, so a weak result can be traced to weak data.

Design

This is a geo-split test. Choose regions that can be targeted separately and whose new-customer counts and existing-customer orders moved together before the test. Split them into treated and control groups, balanced on that history. The control regions are the Holdout Group: they keep the current settings, with existing customers left in prospecting.

Design elementChoiceReason
Unit of randomisationRegion: matched geographic areas that can be targeted separatelyCounts every order existing customers place in an area, through any channel
TreatmentCustomer exclusions on every prospecting campaign and platform; the new-customer goal where it runsA partial exclusion leaves customers reachable through another campaign
ControlCurrent settings, customers left inThe comparison the decision needs
Held constantBudgets, bids, creative, brand and other search campaigns, email and promotionsOnly the exclusion differs
Customer definitionOne definition, from the business's records, in every region and on every platformThe lists, the metric and the guardrail must agree on who is a customer
Primary metricCost per new customer by region, from the business's recordsPlatform reports label conversions by their own lists

The settings follow the site's platform guides, which draw on each platform's help pages. Google Ads Help describes customer data segments that can be used for targeting, observation or exclusion, and Performance Max's customer acquisition goal, which the exclusion guide explains can bid higher for new customers or bid only for them. Meta's Custom Audiences page underlies the guide to audience exclusions and overlap on Meta, which records excluded custom audiences as a control that remains a hard limit under Advantage+ audience. Apply them in the treated regions only, on every prospecting campaign, including Performance Max.

Build the lists once, centrally, as the guide to customer lists across ad platforms describes: one written definition of a customer, a consent and purpose check before any upload, and a refresh schedule. That guide is general direction and not legal advice; the business's own privacy notice and lawful basis decide what may be uploaded. The list is First-Party Data, and an upload that is out of date excludes last quarter's customers and nobody since.

Sample size reasoning

Power comes from the number of matched regions and how closely their new-customer counts and existing-customer orders moved together before the test, not from visitor counts. Write down two numbers in words before launch: the smallest change in cost per new customer that would change the decision, and the largest fall in existing-customer orders the business would accept. The incrementality testing guide explains the region logic: regions chosen so their outcome history moves together, checked in a pre-period, with the number of regions usually small. Use it to check the regions can show both effects. If they can detect the fall in new-customer cost but not a fall in repeat orders the business would care about, the test cannot protect the guardrail that decides, and either more regions or a longer test is needed.

Duration

Start with a pre-period long enough to show the regions moving together. Then run the exclusion for at least one full purchase cycle of existing customers, so their normal repeat orders have had the chance to arrive or not. Avoid promotions, launches and seasonal peaks, which move existing customers for reasons that have nothing to do with prospecting. After the exclusion period, leave a short cool-down before reading repeat orders, because an order prompted by an ad seen late in the period may still be on its way.

Reading the results and the decision rule

Compare the treated regions with the control regions against their pre-period relationship, not raw totals. Read cost per new customer with its interval, then existing customers' orders and margin across all channels, total new customers and the matched share. The question is the Incrementality of the ads to existing customers: the orders that disappear when they stop seeing them. Write the rule before launch:

ResultDecision
Cost per new customer falls and existing-customer orders across all channels hold within the limitKeep the exclusion on every prospecting campaign
Existing-customer orders fall by more than the limitLeave customers in, or give repeat orders a campaign of their own
No reliable change in cost per new customerCheck the matched share; if it is low, fix the list before deciding
Total new customers fall past their limitReview delivery in the treated regions before adopting

The remarketing holdout test is the design for whether ads aimed at existing audiences cause purchases, and the Performance Max uplift test reads new-customer share as a guardrail when the question is whether to add that campaign type at all.

Pitfalls

A stale list that excludes few customers is the first trap. A list uploaded once excludes the customers of that month and nobody since; refresh it on the schedule the plan names, and report the matched share.

Platforms excluding different people is the second. Each matches a different part of the list, so a customer excluded on one may still be reached on another. Read the matched share per platform and note which customers each can see.

Repeat orders read too early is the third. Existing customers buy on their own cycle; a reading inside it shows an exclusion that has cost nothing yet.

Regions leaking through travel or delivery addresses is the fourth. A customer who lives in a control region and sees ads at work in a treated one, or who ships to a different region from where they browse, blurs the comparison. Assign orders to regions the same way in both groups, and prefer regions large enough that most customers live, browse and receive deliveries in one.

Example: an invented bike parts shop tests its exclusions by region (illustrative)

Illustrative example — not a real result

Twindoral Cycles tests excluding existing riders from prospecting by region

Twindoral Cycles, an invented online bike parts shop, runs Performance Max and paid social prospecting. It matches twelve regions on eight weeks of history and applies its customer list exclusion and the new-customer goal in six of them for ten weeks, about one purchase cycle for its riders, followed by a two-week cool-down. Google matches 61% of the list and Meta 55%. Before launch the team agrees that existing riders' orders across all channels may fall by no more than 3% against what the pre-period predicts.

Measure (ten weeks)Control regionsTreated regions
Prospecting spend£24,000£24,000
New customers1,0001,140
Cost per new customer£24.00£21.05
Existing riders' orders against pre-period prediction0%−1.0%

Cost per new customer falls by about £2.95, with an interval of roughly £1.10 to £4.80. Existing riders' orders through email, search and direct visits fall by about 1%, with an interval of roughly −2.6% to +0.6%, inside the limit. Twindoral keeps the exclusion on every prospecting campaign and moves its list refresh from monthly to weekly.

Common mistakes

Exclusion tests usually fail on what they count. The experiment library lists the other paid media designs that read outcomes by region.

  • Reading only the campaign report. It cannot see repeat orders placed elsewhere.
  • Excluding on one platform only. Customers are then reached through another.
  • Using each platform's own new-customer label. Define a customer once, in the business's records.
  • Changing brand search at the same time. Its job includes customers; keep it identical in both groups.
  • Reading before a purchase cycle has passed. The cost of losing repeat orders has not arrived yet.
  • Ignoring the matched share. A poorly matched list tests a weak version of the policy.

Frequently asked questions

Why test by region instead of splitting the customer list in half?

Because the question is what happens to existing customers' orders across every channel once the ads stop reaching them, and that includes orders through email, search and direct visits that no platform attributes. Regions let the business count everything its customers in an area buy, from its own records. A split list would also need every platform to honour the same half, which is hard to guarantee when each matches a different share of the list.

Should brand search campaigns exclude customers during the test?

No. Brand search exists partly to serve customers looking for the business by name, and the site's exclusion guide lists it among the campaigns that should keep reaching them. Search campaigns and brand terms stay exactly as they are in both groups, so the test isolates prospecting.

What if our customer list matches poorly on one platform?

Then the exclusion there removes few customers, and the test measures a weak version of the policy. Read the matched share as a guardrail, report it per platform, and improve the list before relying on the result. A poor match on one platform is a data problem to fix, not evidence that exclusion does not work.

Terms used on this page

Tools to try

Frameworks

Sources

  1. About audience targeting — Google Ads Help
  2. About Performance Max campaigns — Google Ads Help
  3. About Custom Audiences — Meta Business Help Centre

Related