Why a release can break a path without anyone noticing
A fault that appears only on one browser, device or payment method is the kind the diagnosis of an overnight fall in conversion lists as a bug that only some visitors meet and as checkout or payment failing for some customers.
The advice to walk the paths already appears as a single line in each of three prevention routines. The diagnosis above asks for the journey to be walked, after every release, on the browsers and devices that carry the sales. The diagnosis for a mobile gap asks for a walk from landing page to confirmation over a throttled connection. The diagnosis for abandoned forms asks for every live form to be filled in on a phone and a desktop and the entry confirmed to have arrived. The tracking verification checklist asks for a monthly pass over the key journeys and for a run after a release that changes a form, the checkout or the templates the tags rely on. The measurement governance playbook suggests the same monthly verification and holds back a release that alters a purchase, a lead form or a key event until a second person, not the implementer, has verified it. None of them says what the walk consists of. This page does.
The walk has a narrow job. It does not judge the design, and it does not prove the tracking, which a checklist does in the analytics tools. It asks whether each path that earns money still works for the visitors who carry the conversions, ends in the business's own record, reaches the person who has to act, and is counted once.
| Check | When it runs | What it proves |
|---|---|---|
| This walk | After any release that touches a path or what the path shares | The path works for the devices and methods that carry conversions and ends where the business needs it |
| Tracking verification checklist | After tag, pixel, conversion action or banner changes, after a release that changes forms, checkout or shared templates, and monthly | A real action produces the planned event in every system that should receive it |
| Landing page checklist | Before paid or email traffic reaches a page, and again after a redesign or when the form or its destination changes | The page is ready: message, form, proof, speed and tracking |
| A/B testing guide | Before a test launches | Each variant renders, assigns and records as the plan says |
| Evaluating a redesign | Before and after a relaunch | What to record, which templates to test and which rollback rules to agree, so the new site is judged fairly |
| Live error response SOP | After a report that something live is wrong | The error is confirmed, contained, corrected and logged |
Before you start
- The release's change list. What the release touched: templates, forms, checkout steps, scripts, tags, the consent banner, third-party widgets and content such as prices. A developer or agency who can say what changed, and what the changed parts are shared with, saves the guessing.
- The conversion outcomes and the record that proves each. The order system, the booking system, the account database, and the inbox or customer record system that receives enquiries. The page a visitor sees at the end is not the outcome; the record is.
- The site's own breakdowns. Conversions by device, browser, source and payment method; the funnel analysis tutorial builds the device and source segments.
- A convention for test records. The martech guide to test environments and test data covers marked records, invented details and exclusions; agree the marker before the first walk, not during it.
- Real devices. The phones and browsers the breakdowns point to, and a desktop for comparison.
- Access to the far end. The order system, the customer record system, the inbox that receives notifications, the mailbox for confirmations and the analytics debug view.
- A walker who did not build the release, following the script written in the steps below.
- A record of each walk, with dates, screenshots and the release it belongs to.
The steps
List the paths that earn money and where each ends
Write one line for each outcome the site exists to produce: an order, an enquiry or quote request, a booking, a sign-up or trial start, a payment on an account. For each, name the entry points that matter, such as an advertisement landing page, an email link, a search result or a direct visit, and name the record where the outcome has to appear. The guides to sign-up flows and booking flows describe what those journeys contain. Keep the list to outcomes the business counts as success. A page that merely informs is not a path, and a short list is a list that gets walked.
Read the release and mark what it touched
Go through the change list and mark each path as touched, shares something that was touched, or untouched. A touched path is walked in full. A path that shares a template, a script, a tag or the banner with something that changed is walked too, because shared parts are how a change in one place can break another. An untouched path that does not share anything can wait for the scheduled walk. If the change list is vague, treat every path as touched and tighten the list once the developers have named what moved.
Choose the devices, browsers, entry points and payment methods from the site's data
Open the breakdowns and take the combinations that carry the conversions, add the slowest common phone, add each live payment method, and add the entry points the campaigns currently send traffic through, including a link opened inside a social app's in-app browserThe browser built into a social, video or messaging app that opens a tapped link inside the app instead of handing it to the phone's own browser., which may not behave like the phone's own browser. Date the list and keep it short enough to run before the next campaign or email send goes out. Revisit it whenever the mix moves, so it does not become a record of last year's visitors.
Prepare marked test records that cannot be mistaken for customers
Use invented details with the agreed marker, and put the exclusions in place before anything is submitted: the analytics property's internal trafficVisits from the business's own offices, staff, agencies and testers, labelled by an address rule and filtered so they do not count as customers. rules, the report filters, and the audience and upload exclusions. A dedicated test product, a form sent from an address the business controls, or a transaction in the provider's test mode, as the tracking verification checklist suggests, keeps the walk from becoming revenue. Where none of those is possible and a real payment is unavoidable, agree the refund and the marking beforehand. That is general practice, not documentation of any payment provider. Do not use real customers' details.
Write the script for each path
For each path, write the actions in order and the result expected at each one. Include the ordinary awkward inputs that real people supply: a phone number written with spaces, a long name, an address line that wraps, a value typed in the wrong format that the form should explain rather than refuse. Include the moments the release may have changed: the price and the delivery cost shown before payment, the error message when a required field is left empty, the confirmation page after it is reloaded, and what the back button does. The forms guide and the checkout guide set out how errors, costs and payment should behave, so the script can ask for that behaviour by name. One table across the paths is enough to start with:
Path Start Actions to script Where it must end What to count Purchase Each campaign landing page and a direct visit Add an item, change a quantity, choose delivery, pay with each live method An order in the order system with the right items and value One purchase event with the order's transaction identifier Enquiry or quote An advertisement, an email link and the contact page Fill the form with awkward but valid values, leave a required field empty, submit A record where the team works, with its source, and a notification that arrives One lead event for the enquiry, even if the confirmation page is reloaded Booking The booking page from a phone Choose a slot, change it, confirm, cancel A booking in the booking system and a confirmation message One booking event Sign-up The sign-up call to action and the pricing page Create an account, verify it, reach the first screen An account that can sign in, in the right plan One sign-up event Walk every path on every chosen combination
Start each walk from a clean state, in a private window or a clean browser profile, as the tracking verification checklist asks, and check that the consent bannerSoftware that collects, stores and passes on visitors' consent choices for cookies and tracking. appears as a new visitor sees it and can be dismissed with a keyboard. Walk the path as the script says, from each chosen entry point, on each chosen device and browser, and note the time each step takes as well as whether it works: a path that completes slowly on a phone over a mobile connection is a finding. The landing page checklist makes the same request, with a mid-range phone and a mobile connection in place of office wi-fi. Lab impressions are for finding causes, and the speed and mobile guide explains why real-user data, which the Web Vitals documentation describes, is the judge of speed. On any path whose forms or controls the release touched, repeat the keyboard-alone pass from the accessibility guide, which draws on the Web Content Accessibility Guidelines.
Follow each result to the record and to the person who must act
A confirmation on the screen is not the end of the path. Open the order system, the booking system or the customer record system and find the test record with the values that were entered, its source, and its owner or queue. Check the notification and the confirmation message, in the inbox and in the junk folder, because a message that is sent and filed as junk looks, from the visitor's side, like a message that was never sent; the diagnosis of missing order and account emails separates the two. Where the record is a lead, ask whether the person who has to respond can see it: the lead routing QA checklist covers the machinery from form to salesperson, and this step asks only that the lead arrived with its source where the responder works.
Check that each conversion is counted without repeats
In the analytics debug view, confirm that the outcome's key eventA GA4 event marked as important to the business, the property's measure of success. is recorded, since events are the unit of measurement, as the Google Analytics documentation on events describes them. When it fires is a rule of the site's own guides: for a form, after the server accepts the submission and not on the click, as the form tracking guide sets out. Reload the confirmation page and use the back and forward buttons to confirm the count does not move; the confirmation pages guide counts each conversion once and explains why the order's transaction identifier, with deduplicationRemoving double counts when one conversion is reported by more than one source., is what stops a repeat. Use the container's preview mode as the debugging tutorial describes. Once a full day has closed, compare that day's conversions in analytics against the business's own records of that day, as the first check in the diagnosis of a sudden rise in the rate does, and split the comparison by the path and the payment method the release touched. Marked test records stay out of that comparison, because their job was the walk. A debug view that looks right does not prove that the report will agree.
Record the walk and decide what a failure triggers
Write the result by path and by combination: pass or fail, who walked, when, which release, with a screenshot of anything that failed. Log the release in the measurement change log, as that tutorial asks of a release that touches tracked pages or the checkout, and keep the walk record beside it. Then decide what each failure triggers before the next release, not during it. A live page on the path that is wrong for customers is a live error, which the live error response SOP handles. A path that works while its events do not is a measurement fault: the diagnosis of purchases not tracking covers purchases missing for one payment method, and the tracking outage response SOP handles the response. For either, settle beforehand whether the release is reversed at once. The framework on one-way and two-way doors asks how cheaply a decision can be walked back; a two-way door can be acted on, watched on a set date and reversed if the result misses the expectation. A rollbackReturning a changed system, such as a tag container, workflow or page template, to its last working version, planned and rehearsed before the change goes live. is planned before the change goes live, so the previous version and the person who may choose it are named in advance.
Clean up and set the next walk
Remove or mark every test order, lead, booking and account, cancel what must be cancelled, issue the agreed refunds and confirm no marked record leaked into a report, an audience list or a platform upload. Then set the next walk: after the next release that touches a path or what it shares, and on the schedule the key journeys need. Update the walk list from the breakdowns, retire combinations that no longer carry conversions and add the ones that now do. Store the script beside the list so that anyone who did not write it can run it.
What a good record looks like
The record is short and comparable from one release to the next. A table with a row for each path and a column for each chosen combination, filled with pass or fail and a reference to the evidence, is enough. Beside the table, keep the list of what was not walked and why, such as a combination retired from the list or a path judged untouched, because the gaps are what a later investigation needs. Compare the record with the release's change list: every touched path should appear, and every failure should appear with its decision.
A weathervane maker walks its paths after a checkout release (illustrative)
Karduxon Weathervanes walks its paths after a checkout release
Karduxon Weathervanes, an invented workshop that makes weathervanes and finials, takes orders for stock pieces through its shop, quote requests for commissioned pieces through a form and applications from trade buyers through a sign-up. On Tuesday evening a release moves the delivery choice into a new component and updates the payment provider's embedded form. Over the last full month the shop took 1,500 orders: 62% on phones, with three browser and device combinations carrying 88% of orders, card payment on 71%, a wallet button on 24% and bank transfer on 5%.
The walk list takes those three combinations plus the slowest common phone, each of the three payment methods, and entry points from the email newsletter and a social post opened inside the social app. The next morning, before the day's newsletter goes out, a colleague who did not build the release walks the purchase, quote and trade paths from a clean state, using the marked test customer and a test-mode payment.
Two faults appear. On one phone browser the delivery step refuses a valid postcode written with a space, with an error that disappears off screen, so the path cannot be completed. And wallet orders reach the order system with the right items, but their purchase event is missing in the debug view, because the wallet no longer returns the buyer to the confirmation page. The quote and trade paths pass, and the quote request arrives in the team's queue with its source. The walker enters the release and both failures in the change log, the owner follows the failure rule agreed before the release, reverting the postcode rule at once and restoring the wallet return page that morning, and the comparison for the first full day after the fix finds that analytics and the order system agree on that day's real orders, wallet orders included. Without the walk, the first sign would have been a reported conversion rate about a quarter lower, and the shop would have been left asking whether the new delivery component had cost it customers.
The funnel and checkout section sets this routine beside the journeys it protects, from the page where a choice is made to the page after the conversion.
Common mistakes
- Walking only the page that changed. Paths share templates, scripts, tags and the banner, so a change in one place can break another. Walk what the change touches and what it shares.
- Testing on the builder's own device. The visitor's phone, browser and connection are the test. Take them from the conversion breakdowns, not from what is on the desk.
- Stopping at the thank-you page. The outcome is the order, the lead or the booking in the business's own record, seen by the person who must act.
- Using a banner that is already accepted. A warm browser hides what a new visitor meets. Start clean, and check the banner as well as the page.
- Trusting a debug view alone. An event that fires once in the debug view can still be missing or doubled in the report. Compare a closed day's conversions with the business's records once the day has ended.
- Letting test records into the numbers. Mark them, exclude them before the walk and remove them after it.
- Signing off one's own release. The playbook named above asks for a second person to verify a release that touches a purchase, a lead form or a key event, so the builder is not the verifier.
- Keeping no record. Without one, nobody can say later what was checked, on what, and what was decided about each failure.
