Where a test can run: the browser, the server or a separate page
Someone has to decide how each variant will reach the visitors assigned to it. The A/B testing guide describes the two main mechanisms in its section on randomisation and allocation. A client-side tool makes the assignment inside the browser, typically from a hash of a cookie value, and alters the page once it has loaded. Server-side assignment decides before the page is built, so the visitor receives the version they were given as a finished page. The experimentation hub adds a third design to its table of test types: the split-URL test, in which two separately built pages exist at different addresses and visitors are redirected to one or the other.
In all three, the unit of randomisationWhat a test assigns at random to control or variant, such as a visitor, session, account or region; it should match the level at which the metric is counted. is set by what the metric counts, the assignment is sticky for the life of the test, and the plan, the sample and the reading date are fixed before launch. Where a test runs changes what delivery costs, not how the result is read.
| Where the test runs | How the visitor receives the variant | Main costs | Usually suits |
|---|---|---|---|
| In the browser (client-side) | The page arrives as normal; a script assigns the visitor and edits the page after it starts to render | Flicker, a reveal delay on every visit, script work during the visit, winners served from outside the code | Light changes to copy, images and layout on pages where nothing is calculated |
| On the server (server-side) | The server decides before the page is sent and builds the assigned version | Developer time for every variant, and a release for every test | Prices, totals, the steps of a flow, anything the server calculates, speed tests |
| As separate pages (split-URL) | Two separately built pages; visitors are redirected to one of them | Redirect latency, tracking through the redirect, redirects that fail on some browsers | Redesigned templates, and structural changes that in-page edits cannot express |
Where measurement tags run is a separate decision, set out in the comparison of client-side and server-side tagging. The number of visitors a test needs is a question for the guide to sample size and significance, and judging a whole new site one for the guide to evaluating a redesign.
What a test in the browser costs
The appeal of a browser test is speed of launch. Because a script applies the change on top of the page, it does not wait for the site's own release, and a new headline or image can be built and checked without touching the templates. The costs are less visible, and four of them deserve counting before a test is placed in the browser.
Flicker. The testing pitfalls guide explains why a browser tool flickers: it swaps in the variant once rendering is already under way, so on a slow device the original is briefly visible before being replaced. The flash looks like a fault, and it changes how people behave in the variant arm alone, so the comparison is no longer of the change by itself but of the change plus a flash against no flash. The guide's remedies are moving the change to the server, applying it before first paint or making the change simpler so it lands sooner, with a restart if flicker was common during the run. Checking for flickerThe brief flash of the original version before a browser-side test script applies the variant, which turns a result about the change into one about the fault. on a slow phone before launch means the problem is found before any visitors are counted rather than after.
A delay on every visit. The speed and mobile guide points out that a browser-side script that keeps the page hidden while it chooses a version makes every visit wait, control visits included, and asks for that delay to be as short as the tool allows wherever such a tool is used. Because both arms carry the delay, the comparison still looks fair, and the result never shows that the tested pages became slower for everyone. The guide draws on web.dev's documentation of Core Web VitalsGoogle's user experience metrics., listed in this page's sources, which measures loading, responsiveness and visual stability from real users, and a page held back by a script cannot show its largest element until the script lets it. Content injected by tags is also among the causes of layout shift the guide lists, so a variant that inserts a banner or a block above where the visitor is reading can move the page as it appears.
Work on the main thread. Testing tools appear in the slow interaction responsiveness diagnosis as one of the kinds of added script that share the browser's main thread with the site's own code and keep it occupied. As general practice, a test script belongs only on the pages where tests are actually running.
Winners outside the code. A browser test's winner exists only as an edit applied by the script. The tutorial for shipping a winning variant rebuilds winners in the site's own templates, because a script left serving a change adds weight, can flicker and competes for the main thread, and because the change then lives where the next developer will not look for it. Some of the build effort a browser test saves at the start is spent later, when the winner has to be built again properly.
There is also a limit to what a browser test can honestly change. A script can alter what the page displays; it cannot alter what the server calculates, stores or charges. As general practice, a test of a price, a delivery charge, a total, availability or the steps of a booking or checkout flow should not be run by editing the page in the browser, because what the visitor reads and what the system does can then disagree.
What a server-side test costs
A server-side test moves the decision to the point where the page is built. The server assigns the visitor, builds the version that visitor was given and sends it as a finished page. The A/B testing guide states the advantage plainly: there is nothing to flicker. Nor is there a script holding the page back while it decides, which is why the speed guide names server-side or edge-based variation as the way to avoid the reveal delay.
The cost is engineering time. Every variant is written into the site's code by a developer, reviewed like any other change and released through the site's normal process, and that time competes with everything else the developer could be building. For a programme with few developers it decides how many tests can run, which is a reason to spend server-side tests on the changes that need them rather than on every headline.
Three further things become the team's own responsibility rather than a tool's.
- Assignment and stickiness. The site has to assign each unit at random, remember the assignment and serve the same version on every visit. Usually the unit is a visitor, recognised by cookie or by login, or an account where colleagues should see the same version, following the A/B testing guide.
- The record of who saw what. Because the server assigns before rendering, analytics needs to be told, at the moment the visitor actually reaches the tested page, which test and which version they saw. That record is the exposure event described in the measurement section below.
- Checking before launch. A variant built in code is checked in a staging environmentA private copy of a website or tool where changes are tested before release, kept behind access control and out of search and production analytics. before any visitor sees it. The guide to test environments and test data passes on the advice to test on a staging copy first where one exists, especially anything touching checkout or forms, and warns that a staging site sending to the production analytics property is not a test environment at all.
The return comes at both ends. During the run, the variant behaves as the shipped version would, so the result measures the change rather than the change plus its delivery. After a win, the variant already exists in the site's code, so of the shipping tutorial's steps only the device and tracking checks, the release log entry, the removal of the losing version and the closing of the test's record remain.
Split-URL tests for whole pages and templates
Some changes are too large to express as edits to an existing page. The experimentation hub's table of designs reserves the split-URL test for redesigns and structural changes of that kind: two separately built pages, with visitors assigned at random and redirected to one of them. The guide to evaluating a redesign uses it to test the templates that carry most conversions, such as product, pricing, basket and lead-form pages, before the rest of a new site moves.
The hub and the redesign guide name the risks, and each one has a check.
- Redirect latency. The redirected arm loads later than the arm served directly, a disadvantage that disappears once the new version is live. Measure the delay in each arm.
- Tracking through the redirect. The assignment, the landing on the redirected page and the conversion all have to be recorded for both arms. As general practice, check that campaign parameters survive the redirect, so that the redirected arm's visitors keep the source they arrived from.
- Redirects that fail on some browsers. The testing pitfalls guide gives this as a common mechanism behind a sample ratio mismatchA persistent gap between the share of visitors an A/B test was meant to assign to each variant and the share each actually received.: visitors whose browser fails the redirect fall back to the control, and the arms stop being comparable. Check the split every day, by device and by browser, and try the redirect on every browser that carries a meaningful share of the traffic before launch.
A split-URL test measures the net effect of everything that differs between the two pages: whether the new template beats the old one, which is the question the business faces, and not which of its changes made the difference.
Where a template cannot be split at all, because the platform can only serve one version at a time, the redesign guide's fallback is a before-and-after comparison over matched weeks, with the prediction written before launch. Where the old and new versions can run side by side but a split-URL test is impractical, a staged rolloutReleasing a change to a growing share of visitors, pages or regions in steps and reading each before the next; a test only if groups are random and concurrent. exposes the change to a growing share of visitors, and the hub is precise that it counts as a controlled comparison only when the groups are random and concurrent.
Choosing by the change being tested
The useful question is not which method the programme uses but where each planned test should run. Sort the backlog by what each change does.
| The change being tested | Where it usually belongs | Why | Check before it counts |
|---|---|---|---|
| Headline, copy, image or button wording on a campaign landing page | The browser, if it passes a flicker check | Light and visual, applied quickly, nothing calculated | Load it on the slowest common phone over a throttled connection; time the reveal |
| A rearranged or inserted section on a content or product page | The browser only if it applies before the visitor sees the page; otherwise the server | Blocks moved after render shift the layout and are the changes most likely to flash | Watch for layout shift as well as flicker |
| A price, discount, delivery charge, total or availability message | The server | What the page shows must match what the system charges and allocates | A full order or booking in staging in each arm |
| The steps, fields or order of a checkout, booking or application flow | The server | Validation, payment and confirmation live in the site's code, and a flash where doubt peaks looks like a fault | The tracking walk in each arm, with every payment method |
| A redesigned template or a whole new page | A split-URL test | Too large for in-page edits | The redirect on every major browser; the split by browser each day |
| Faster loading or responsiveness | A server-side split, always | A browser script slows both arms and may wipe out the very difference being measured | Field speed recorded in each arm |
| A change that cannot be split-URL tested | A staged rollout to a random share if both versions can run at once; otherwise a before-and-after read | A split is impractical, or the platform can only serve one version at a time | Random, concurrent groups, or a prediction written in advance |
Two rows are not matters of judgement. The page speed impact test makes a server-side split a requirement rather than a refinement for any test of speed, because a browser-side tool adds its own script and changes the page late, slowing both arms and hiding the very difference the test exists to measure. And anything the server calculates belongs on the server, for the reason given above.
The checkout row draws on the checkout optimisation guide, which describes the payment step as the point where doubt peaks. A shopper who sees the order summary change in front of them there has a reason to stop.
For the rows that leave a choice, four questions settle most cases:
- Does the change alter anything the server calculates, stores or charges? If so, the server.
- Can the change be applied before the visitor sees the page, on the slowest device that matters? If not, the server, or a simpler change.
- Is the page one where a flash or a moving element would look like a fault? Checkout, booking, payment and account pages are, and tests there belong on the server.
- If the variant wins, who will build it into the site, and by when? A browser test with no answer to that question produces a winner that stays in the script.
Measuring and checking a test however it is served
Moving a test to the server removes flicker and the reveal delay; it removes none of the measurement work. Every test, wherever it runs, goes through the same gate. First in that gate is an exposure eventAn analytics event fired when a visitor sees a test arm, carrying the test and variant, so each arm's behaviour can be counted outside the testing tool. sent when the visitor actually sees the tested version.
- An exposure record in the business's own analytics. The GA4 tutorial for A/B test analysis fires an exposure event, with the test and the variant as parameters, at the moment a person sees the arm, registers the variant and builds the comparison per arm in explorations. For a server-side test, send it when the tested page is actually shown rather than at the moment of assignment; for a split-URL test, send it from the page the visitor landed on after the redirect.
- A daily split check. The same tutorial compares each day's share per arm and treats a gap that persists as a sample ratio mismatch, to be found and fixed before anything is read. Analytics counts fewer people than the assignment does, because consent choices and blockers stop some tags; when both arms lose roughly the same proportion the comparison survives, and when one arm loses more than the other it is dropping events.
- The tracking walk in each arm. The A/B test launch checklist asks for every variant to render correctly on the main browsers and devices, for the original not to flash, for the primary metric to record in every variant and for the variant name to reach analytics. Walk the conversion path in both arms, whatever the delivery method, before any of the test's data counts.
- Speed as a guardrail. The A/B testing guide lists page speed among the guardrails a variant must not worsen. Where the delivery method could itself change speed, as it can for any browser test and for the redirected arm of a split-URL test, record field speed in each arm, as the page speed impact test does for its own guardrail.
- The same statistics. The sample a test needs depends on the baseline rate and the smallest effect worth shipping, not on where the test runs, and the result is read once, at the planned sample.
Who decides, builds and signs off
The experimentation programme playbook divides the work into roles. The experimentation lead owns the programme and the roadmap. The analyst owns test design and results and signs off that a result is reliable. The developer builds the variants and their targeting, wires the test events into tracking and runs the technical checks before launch. The designer owns the variant experience, and the channel owner owns the page or journey being tested, approves tests in it and implements winners. Where a test runs touches every one of those roles, and it is cheapest to decide at the design stage.
A RACI matrix makes the division explicit. Its rows are decisions, each with exactly one accountable role, so nobody is left assuming that someone else checked the redirect or rebuilt the winner. A version for a programme that uses all three methods might start like this.
| Decision or task | Experimentation lead | Analyst | Developer | Designer | Channel owner |
|---|---|---|---|---|---|
| Where each test runs | Accountable | Consulted | Consulted | Informed | Consulted |
| Building a browser variant | Informed | Informed | Accountable | Responsible | Informed |
| Building a server-side or split-URL variant | Informed | Informed | Accountable | Consulted | Informed |
| Flicker, layout and speed checks before launch | Informed | Consulted | Accountable | Responsible | Informed |
| Exposure event, tracking walk and daily split check | Informed | Accountable | Responsible | Informed | Informed |
| Declaring the result | Informed | Accountable | Informed | Informed | Consulted |
| Shipping the winner in the site's code and removing the test | Informed | Consulted | Responsible | Informed | Accountable |
The cells are a starting point; on a small team one person may hold several. Two are worth keeping wherever the table ends up. The analyst declares results, because the playbook's quality gate depends on nobody declaring their own winners. And the shipping row carries a date as well as an owner, so that any winner still served by a script has a planned end.
A van hire firm moves its booking tests to the server (illustrative)
Paxtergund Van Hire sorts its tests by where they run
Paxtergund Van Hire, an invented van rental firm, runs every test through a client-side script, including tests on its four-step booking flow and its price summary. Recordings show the old price summary on screen for around 0.6 seconds on older phones before the variant replaces it, and field data shows the script's reveal delay adding about 0.4 seconds to the load of the largest element on every booking page, in both arms. Two winners from earlier tests, a shorter extras step and a reworded insurance panel, are still served by the script five months after they were declared.
The firm sorts its next 14 planned tests by the change each makes:
| Kind of test | Planned tests | Where it now runs | Built by |
|---|---|---|---|
| Headline and image tests on campaign landing pages | 6 | Browser, after a flicker check on the slowest common phone | Designer, reviewed by the developer |
| Booking steps, prices and anything the server calculates | 5 | Server-side assignment | Developer, checked in staging |
| A redesigned quote page | 1 | Split-URL test, redirect tried on 5 major browsers | Developer |
| Speed work on the booking pages | 2 | Server-side split, field speed per arm | Developer |
It writes a RACI table with the analyst accountable for every result and the experimentation lead accountable for where each test runs, removes the script from the booking pages and rebuilds the two old winners in the site's code over 3 weeks. The largest element on the first booking step then loads about 0.4 seconds sooner in field data. The first server-side test, a price summary that shows the deposit on its own line, runs for 4 full weeks to its planned 9,200 visitors per arm, shows no flicker in any recording and ends inconclusive, with an interval from minus 2% to plus 6% on completed bookings per visitor. It is logged as inconclusive and not rerun.
Common mistakes
- Choosing one method for the whole programme. Place each test by the change it makes.
- Testing prices or flow logic in the browser. What the page shows and what the system does can disagree.
- Treating flicker as cosmetic. It changes behaviour in the variant arm only; fix it, and restart if it was common.
- Ignoring the reveal delay because both arms carry it. The result hides it, and every visitor pays it.
- Deciding where a test runs after it is built. Settle it at design, with the developer consulted.
- Running a speed test through a browser script. The script slows both arms and can erase the difference.
- Assuming a server-side test needs no tracking checks. The exposure event, the daily split and the tracking walk apply to every test.
- Leaving winners in the script. Name who rebuilds them, and by when, before the test launches.
