Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Marketing Analytics › Google Analytics 4

Analysing A/B Tests in GA4

Send A/B test exposure to Google Analytics, register the variant, check the split, and read the primary metric and guardrails for each variant in explorations.
Tutorialadvanced11 min readUpdated

Why analyse a test in analytics at all

An A/B test is usually run by a testing tool that assigns each visitor to an arm, applies the change and reports the goal it was configured with. That report answers one question. Analytics answers three more. It shows behaviour the tool was never told about, such as basket additions, later visits and revenue. It gives a second, independent count of the outcome, which is the fastest way to notice that a conversion tag behaves differently in one arm. And through the export it lets the test be joined to the order or lead system, where the outcome the business cares about actually lives.

What analytics does not do is the inference. Explorations will show that one arm converted at a higher rate than the other; they will not say whether that difference is larger than chance. That judgement comes from the method the plan fixed in advance, and Evan Miller's essay on how not to run an A/B test shows why it must be fixed in advance: a team that checks repeatedly and stops at the first good-looking moment will declare winners that do nothing.

SystemWhat it is for in a testWhat it should not be used for
Testing toolAssignment, applying the change, its own goal and statisticsThe full picture of behaviour in each arm
GA4 interfaceExposure counts, the split check, the primary metric and guardrails per armSignificance; exact counts for a close decision
BigQuery exportExact, unsampled counts per arm and the join to orders or leadsDaily monitoring by people without SQL
Order or lead systemThe outcome as the business records itKnowing which arm a customer saw, unless analytics tells it

Before you start

  • A written test plan. Hypothesis, unit of randomisation, primary metric, guardrail metrics, planned sample size, duration and the analysis method, as the A/B testing guide sets out. This tutorial implements the plan; it does not write it.
  • Access. Edit and publish rights on the Tag Manager container, and a property role that can create custom definitions and explorations.
  • The testing tool's documentation for how it tells the page which arm a visitor received: a callback, a data layer push or a readable value. Follow the vendor's own documentation for this part.
  • The tracking plan open, so the exposure event and its parameters are added as a row before anything is built; the events and parameters guide describes the naming rules.
  • The BigQuery export linked well before launch if the decision is a large one. The BigQuery export basics guide notes that nothing is backfilled, so a link created mid-test misses the first days.
  • Working knowledge of explorations, segments and funnels, from the explorations tutorial.
  1. Agree the metric and its unit before launch

    Translate the plan's primary metric into the exact terms analytics will count. Name the numerator (for example, users who purchased after their first exposure), the denominator (users exposed to the arm), the event behind each, whether events or users are counted, and the window. The unit must match the randomisation: if the tool assigns visitors, the analysis counts users, not sessions, because a session-level denominator lets one person count several times. Write the guardrails the same way, and name the method that will turn the counts into a decision, whether that is the testing tool's statistics or a calculator. Put all of it in the test record before launch, so nobody chooses a metric after seeing which one looks best.

  2. Send an exposure event with test and variant

    Fire one event when a visitor actually sees an arm, in both arms including the control, carrying a test identifier and a variant identifier. The moment matters: an event fired on assignment, on a page the change does not touch, adds people who never saw anything and dilutes the difference. An exposure event sent when the arm is shown records only people who could have reacted to the change. Have the testing tool's callback push to the data layer, and build a custom event trigger and a GA4 event tag in the container that read both keys as event parameters. Use short, controlled values agreed in the tracking plan, never anything typed by a visitor.

    window.dataLayer = window.dataLayer || [];
    window.dataLayer.push({
      event: "experiment_exposure",
      experiment_id: "pdp_short_2026_11",
      variant_id: "b"
    });
    
  3. Register the variant for reporting

    Parameters that are not registered do not appear in reports or explorations, and the events guide explains that registration is not retroactive. Before launch, register experiment_id and variant_id as event-scoped custom dimensions. Event scope fits because the events guide reserves it for values that describe the event they travel with, and an exposure is exactly that: this person saw this arm at this moment. Reuse the same two parameters for every future test rather than registering new ones each time, which keeps the property inside its documented limits on custom definitions. If a report also needs the arm on later events, a user property can carry it, but it holds one value at a time, so give each test its own name and retire it afterwards.

  4. Verify exposure in DebugView

    Use the testing tool's preview or forcing option to load each arm in turn, with Tag Manager preview running. In Tag Assistant, confirm the data layer push arrives with both keys and that the GA4 tag fires once on it. In DebugView, confirm the event name and both parameter values for each arm, as the debugging tutorial describes. Then walk to the outcome in each arm and confirm the purchase or lead event fires once with the same parameters as outside the test. The test pitfalls guide lists broken tracking as the commonest cause of a confident wrong answer, and this walk is where it is caught, before any visitor is in the test.

  5. Check the split every day

    Build a free-form exploration with variant_id in rows, filtered to the test's experiment_id, with users and the exposure event count as metrics and date as a second dimension. Compare each day's share per arm with the planned split, overall and by device category and browser. A persistent gap is a sample ratio mismatch, which the pitfalls guide treats as a defect that removes visitors from one arm selectively, so stop, find the mechanism, fix it and restart. Compare the exposure counts with the testing tool's assignment counts too. This daily look is monitoring, not reading the result: it checks that the experiment is working, and it never looks at the primary metric.

  6. Build the variant comparison in explorations

    Create one user segment per arm: users who had an experiment_exposure event with this experiment_id and this variant_id. Then build a closed funnel exploration whose first step is the exposure event narrowed to the test by its parameter and whose second step is the agreed outcome, indirectly followed, and apply the two segments as comparisons. A closed funnel counts only people who completed the first step, so a purchase made before the exposure is not counted, which is the trap a plain user segment falls into. Set the date range to the test's run, check the sampling indicator before reading, and name the exploration with the test identifier.

  7. Add the guardrail metrics

    Each guardrail from the plan gets the same treatment: a defined numerator and denominator, read per arm, from the first exposure onward. Revenue per exposed user comes from a free-form table with the two segments as comparisons and purchase revenue and users as metrics; refunds come from the refund event where the shop sends it; engagement and error events come from their own events. Keep the guardrails few and named in advance. A variant that wins the primary metric and breaches a guardrail goes back to the decision makers rather than straight to rollout, which is the rule the A/B testing guide sets.

  8. Read the result with the method agreed before launch

    When the planned sample is reached, and not before, take the counts per arm from the funnel: users exposed and users who reached the outcome. The conversion rate calculator turns them into rates; the significance judgement comes from the method named in the plan, applied as the sample size and significance guide explains. Read the guardrails alongside. Record the counts, the rates, the decision and any caveat, such as a sampled exploration or a consent gap, in the experiment results report, which also records the check that the variant parameter arrived.

  9. Confirm in the BigQuery export when the decision is large

    For a change that is expensive to reverse, recount in the export, which holds every event without the sampling and thresholds the interface applies. Take each user's first exposure to the test, keep the arm they saw, and count outcomes after that timestamp. Users who appear in both arms point at an assignment fault and should be investigated, not quietly dropped. Where the purchase carries a transaction identifier, the same query can join to the order system, which is how returns or cancellations become a guardrail the interface cannot show.

    WITH exposures AS (
      SELECT user_pseudo_id,
        (SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'variant_id') AS variant_id,
        MIN(event_timestamp) AS first_seen
      FROM `project.analytics_property.events_*`
      WHERE _TABLE_SUFFIX BETWEEN @start_date AND @end_date
        AND event_name = 'experiment_exposure'
        AND (SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'experiment_id') = 'pdp_short_2026_11'
      GROUP BY user_pseudo_id, variant_id
    )
    SELECT e.variant_id,
      COUNT(DISTINCT e.user_pseudo_id) AS exposed_users,
      COUNT(DISTINCT IF(p.event_timestamp > e.first_seen, p.user_pseudo_id, NULL)) AS purchasers
    FROM exposures e
    LEFT JOIN `project.analytics_property.events_*` p
      ON p.user_pseudo_id = e.user_pseudo_id AND p.event_name = 'purchase'
      AND p._TABLE_SUFFIX BETWEEN @start_date AND @end_date
    GROUP BY e.variant_id;
    
  10. Close the test and clean up

    When the decision is made, end the test in the testing tool and remove the losing arm's code, so the exposure event stops for that test identifier. Keep the event and its two registered parameters for the next test. Rename or archive the explorations and segments with the test identifier and the result, and retire any user property created for this test. Update the tracking plan row with the test's end date. Write the outcome into the results report and the backlog, so the next hypothesis in the experimentation loop starts from what this one learned rather than from memory.

A test read in GA4 (illustrative)

Illustrative example — not a real result

Tidewren Mattresses tests a shorter product page

Tidewren Mattresses, an invented mattress retailer, tests a product page with the specifications collapsed into tabs (arm b) against the current long page (arm a), on a planned even split for four full weeks. The primary metric is users who purchase after their first exposure, divided by users exposed; the guardrail is the share of orders returned, read from the order system through the export.

Row labelArm a (control)Arm b (shorter page)
Users exposed (GA4)18,42018,310
Share of exposed users50.1%49.9%
Users who purchased after exposure412455
Order rate2.24%2.48%
Orders returned within the order system's window31 (7.5%)36 (7.9%)
Visitors assigned (testing tool)21,69021,560

The daily split stayed between 49.6% and 50.4% for arm a, so no sample ratio mismatch. The testing tool counted about 18% more visitors than GA4 in both arms. The team traced the gap to consent: the tool assigned every visitor, while the analytics tag, under the site's basic consent mode implementation, sent nothing for visitors who declined analytics. Because the gap was the same proportion in each arm, the comparison held. At the planned sample the testing tool's statistics, the method named in the plan, judged the difference in order rate significant at the agreed level. The returns guardrail rose by 0.4 points, inside the tolerance the plan had set at one point. An export recount matched the interface to within a few users, and the shorter page was rolled out with a note to watch returns for two further months.

When analytics and the testing tool disagree

Two systems counting the same test will never agree exactly, and they do not need to. What matters is that any gap is the same in both arms. The table lists the usual causes.

What you seeLikely causeWhat to check
Analytics shows fewer people in both arms by a similar shareConsent choices or blockers stop the analytics tag for some visitorsConsent configuration; the gap's share per arm
Analytics shows more outcomes than the toolThe tool counts its goal inside its own window or session; analytics counts every later outcomeEach system's definition of a conversion
The gap differs between armsA tag, trigger or page element behaves differently in one armWalk the outcome path in each arm in preview
Exposure counts in analytics exceed the tool's assignmentsExposure fires on every view rather than once, or on pages outside the testTrigger conditions; count users, not events
Rates differ although counts look closeDifferent denominators, users in one system and sessions in the otherThe metric definition written in the first step
Interface and export disagree slightlyEstimated user counts and thresholds in the interfaceTreat the export as the exact record

Common mistakes

  • Firing the exposure event on assignment. People who never saw the change dilute the difference until it disappears.
  • Sending the event from the variant only. Without a control exposure there is no denominator for the control arm.
  • Registering the dimension after launch. Registration is not retroactive in the interface; the first days of the test are missing from every exploration.
  • A plain user segment for the outcome. Purchases made before the exposure are counted; a closed funnel or a sequence condition fixes it.
  • Reading the primary metric daily. Checking the split is monitoring; deciding from the primary metric before the planned sample is the peeking that inflates false winners.
  • Treating a difference in rates as a result. Explorations supply counts; the agreed method supplies the judgement.
  • Session denominators for a visitor-level test. One person counted several times in one arm.
  • Ignoring the sampling icon. A sampled exploration is a poor basis for a close call; recount in the export.
  • Leaving finished tests running in the container. Old exposure events clutter the next test's analysis.

The GA4 hub places this tutorial among the other GA4 guides, and the explorations tutorial covers the funnel and segment techniques used here in more depth.

Frequently asked questions

Can GA4 tell me whether my A/B test result is statistically significant?

No. Explorations give you counts and rates for each arm: users exposed, users who completed the outcome, revenue and the guardrails. Whether the difference between two rates is larger than noise is a separate calculation, made with the method the test plan named before launch, which is usually the testing tool's own statistics or a calculator built for the purpose. The sample size and significance guide explains that method; GA4's job is to supply clean numbers to it and to show what else each arm did.

Should the variant be sent as an event parameter or as a user property?

Send it as a parameter on an exposure event and register it as an event-scoped custom dimension; that records exactly when each person saw which arm and lets several tests run at once without overwriting each other. A user property attaches the arm to every later event from that user, which is convenient for some reports, but it holds one value at a time, so two concurrent tests fight over it. If you add a user property as well, give each test its own name and retire it when the test closes.

Why does GA4 show fewer people in the test than the testing tool does?

The two systems count different things. The testing tool counts every visitor it assigned, under its own identifier; analytics counts the people whose exposure event reached the property, which excludes visitors whose consent choices or blockers stopped the tag, and it groups events into users with its own identifiers. A gap of similar proportion in both arms is normal and leaves the comparison intact. A gap that differs between the arms means something in one arm is losing events, and the test cannot be read until the cause is found.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Sources

  1. How Not To Run an A/B Test — Evan Miller
  2. [GA4] About events — Google Analytics Help
  3. [GA4] Get started with Explorations — Google Analytics Help
  4. [GA4] Set up BigQuery Export — Google Analytics Help

Related