Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Marketing Technology › The Marketing Stack

Test Environments and Test Data for Marketing Tools

How to test changes to marketing tools before customers see them: where tests run, preview and test accounts, invented test records, reports and clean-up.
Guideintermediate9 min readUpdated

Why testing in live tools goes wrong

Marketing teams test all the time. They submit the form to see if it works, place an order on the new checkout, send the email to themselves, walk a contact through a workflow. Most of these tests happen in the live tools, because that is where the change is and there is often nowhere else. The problems come from what the test leaves behind.

A test order is counted as revenue in analytics and in the warehouse. The same order fires a purchase event that the ad platforms receive and learn from. A test contact enters a nurture workflow and syncs to the CRM, where a salesperson calls it. A test segment built from real customers sends a draft email to a few thousand people. A colleague tests a form with their own details and becomes a lead. None of this is malicious, and all of it is avoidable with a few rules applied across every tool, not just one.

The site already covers testing inside individual tools. The guide to debugging with preview mode tests tag changes before they are published; the automation governance guide tests workflows with seed contacts; the tracking verification checklist checks a live set-up. This guide covers what sits between them: deciding which changes need testing, where, with what data, and how to keep the test from becoming part of the business's numbers.

Which changes need a test first

Not every change needs a separate test. The useful line is the one the one-way and two-way doors framework draws: a two-way door can be walked back through cheaply, so it can be tried live and corrected; a one-way door cannot, so it is tested before anyone walks through it. Applied to a marketing stack, the one-way changes are the ones whose effects leave the business's control.

ChangeDoorWhy
A message sent to customers (email, text, in-app)One-wayA sent message cannot be recalled
Data sent to ad platforms (conversions, audiences, uploads)One-wayThe platform records it and its bidding learns from it
A deletion, merge or bulk update of recordsOne-wayThe previous state is gone unless it was exported first
A change to consent handling or tag firingOne-wayData collected without the right permission cannot be uncollected
A new tag, trigger or event on the siteMostly one-wayIt changes what is collected and sent from the moment it is published
Report layout, a dashboard filter, a draft segmentTwo-wayCan be corrected with no effect outside the business
Copy or design in a draft that is not yet liveTwo-wayNothing has left the tool

The table is general practice, not a rule set for any tool. Its use is to make the team ask, before each change, whether anything leaves the business when it goes live. If it does, it is tested first, somewhere customers and platforms cannot see.

Where a test can run

There are four places a test can run, from safest to least safe.

Preview modes. Google documents Tag Manager's preview and debug mode for testing container changes before they are published. As the debugging guide describes, preview loads the draft workspace in the tester's browser and shows which tags fired and why, with the state of the data layer at each event, while visitors keep receiving the published version. Shared preview links expose the draft changes to whoever holds them, so they are shared with people rather than posted in channels. The container governance guide adds the working rules: one change per workspace, described before preview, reviewed by someone else before publishing. Preview is the default place to test a GTM change, but a tag tested in preview can still send real data to a platform if the tester completes a real conversion, so the test records below still apply.

Separate test accounts, properties or workspaces, where a tool offers them. Some tools provide a sandbox, a test account or a separate property; many do not, and this guide makes no claim about which. Where one exists, it is the place for anything that would otherwise send messages or data. Where a candidate tool offers none, the choosing and evaluating tools guide treats that as a finding to score, and the guide to choosing an email platform treats a refused trial or sandbox account the same way.

A staging copy of the site. Where the business has one, the container governance guide recommends testing changes there first, especially anything touching checkout or forms. A staging site that sends to the production analytics property or the live ad accounts is not a test environment; check where its tags send before relying on it, because a staging site sending to production is one of the causes of test traffic the data quality guide lists.

The live tools, with marked test records. For most marketing automation, CRM and email changes, the test runs in the live tool because there is nowhere else. That is workable if the test records are unmistakable, the exclusions are in place before the test starts, and clean-up is part of the test.

Test records that cannot be mistaken for customers

Test data is invented, not copied from customers. That follows the data minimisation principle as the GDPR glossary entry describes it: hold only the personal data a purpose needs, and a test does not need a real person. Copying customer records into a test account creates another copy of personal data with no purpose beyond the test, and a test that needs real data, such as a trial of a new tool against the business's own consent fields, goes through the privacy review in the marketing tool onboarding SOP first.

Make every test record recognisable at a glance and by a filter. A consistent marker in the name, a dedicated email domain or address pattern the business controls, a test flag where the tool has a field for it, and test orders placed with a marked product, coupon or customer account. The marker matters more than the specific convention, as long as it is written down and used every time. Keep a small set of permanent test records rather than creating new ones for every test, which makes them easier to exclude.

The pages that own specific test records stay the reference for them. The automation governance guide's seed contacts are permanent fixtures with names that make them unmistakable, excluded from reporting and from sync to sales. The tutorial on building a closed-won workflow creates a test company, test contacts and a test deal handled as a test rather than uploaded as revenue, and the email automation flow launch checklist walks test contacts through every branch. This guide asks only that every tool's test records follow one convention.

Keeping test data out of reports and platforms

A marker only helps if something filters on it. For each place test data could count, write the exclusion and put it in place before the test runs. Reports and dashboards exclude marked records and orders. Warehouse pipelines drop or flag them before they reach reporting tables. Audience rules and customer list uploads exclude anything marked. Syncs to the CRM skip test contacts, or flag them so sales never sees them as leads. And conversion routes to ad platforms skip marked orders.

Analytics, whether GA4 or another tool, needs the same care. The data quality guide lists internal and test traffic among the common failures, with symptoms such as events with test names and conversions from staff, and its weekly routine checks that the internal traffic filter is active and looks for events with test prefixes. The guide to keeping personal data out of analytics is a reminder that a test form submitted with real details can put those details into the property, which is another reason test records use invented ones. Do not rely on memory for any of this: the exclusions are part of the stack's configuration, recorded with the integrations they apply to.

Cleaning up and recording each test

Every test ends with clean-up. Remove or mark test orders and contacts the test created, take test records out of any workflow they are still in, delete test segments and draft sends, close shared preview links, and check that nothing marked reached a report, an audience or a platform. If something did, record it and correct it: note the affected period in the measurement change log so a later reader knows why the numbers moved.

Then record the test itself: what changed, where it was tested, which test records were used, what the result was and who approved going live. The record belongs with the change, in the workspace description, the ticket or the change log the tool already uses. A test without a record is repeated from scratch next time, and a test that went wrong without a record is repeated in the same way.

A ticketing company separates its tests (illustrative)

Illustrative example — not a real result

Gindlemaze Tickets moves its tests out of the live numbers

Gindlemaze Tickets, an invented event ticketing company, changes its checkout to add a seat-selection step. Over two days, 3 people test it by buying real tickets with real cards and refunding them. The 23 test orders appear in analytics as £1,840 of revenue, and the purchase events reach both ad platforms the company uses, which count them as conversions from the campaigns that brought the testers to the site. The weekly report shows a jump that the marketing lead nearly attributes to a new campaign.

The team changes four things. Tag changes are now tested in Tag Manager's preview mode, with a staging copy of the checkout that sends to a separate test analytics property. Every test order is placed through one test customer account, with a marked name and an email address on a domain the company owns, and buys a marked test event. Reports, the warehouse pipeline, the audience rules and the conversion routes to the ad platforms all exclude that account and that event. And every test now ends with a clean-up step and a line in the change log.

The next checkout change produces 31 test orders, all on the staging site or through the test account. None appears in the weekly report or in either ad platform.

Common mistakes

  • Testing one-way changes live. A sent message, an upload to an ad platform or a deletion cannot be walked back. Test them where customers and platforms cannot see.
  • Using real customers as test data. It creates another copy of personal data with no purpose beyond the test; invent the records instead.
  • Test records that look like customers. Without a marked name and domain, nobody can filter them out or recognise them later.
  • Adding exclusions after the test. By then the order has reached the report and the conversion has reached the platform. Put the exclusions in first.
  • A staging site that sends to production. It is not a test environment until its tags send somewhere separate.
  • Sharing preview links in open channels. Shared previews expose draft changes to anyone holding the link.
  • No clean-up step. Test contacts left in workflows keep receiving messages and keep appearing in counts.

Frequently asked questions

Can we test with real customer records if we are careful?

It is better not to. Invented test records do the job of a test without exposing anyone, and they follow the data minimisation principle as the GDPR glossary entry describes it: hold only the personal data a purpose needs, and a test does not need a real customer. Where a test or trial genuinely needs real data, such as checking how a new tool handles the business's own consent fields, it goes through the privacy review in the marketing tool onboarding SOP first, and the data used is the smallest sample that answers the question.

Does every marketing tool have a sandbox or test environment?

No, and this guide makes no claim about what any particular tool offers. Tag Manager's preview mode is documented by Google for testing container changes before they are published. For other tools, a separate test account, property or workspace is an option only where the tool provides one; the choosing guide treats a candidate with no trial, sandbox or test data as a finding to score. Where no separate place exists, the test runs in the live tool with marked test records and a clean-up step.

How do we stop test data appearing in reports?

Mark it so it can be recognised, then exclude it everywhere it could count. Test records get a marked name and email domain; test events and orders carry a marker the reports and pipelines can filter on; and audience and upload rules exclude anything marked. The data quality guide lists internal and test traffic among the failures to check for, and its weekly routine looks for events with test prefixes, which is how a gap in the exclusions is caught.

Terms used on this page

Frameworks

Where this page is used

Sources

  1. Preview and debug containers — Google Tag Manager Help
  2. Regulation (EU) 2016/679 (GDPR) — EUR-Lex

Related