Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Conversion Rate Optimisation › Funnel & Checkout

Checking Conversion Paths After a Release

How to check conversion paths after a release: choose paths, devices and payment methods from site data, walk each to the record and the count, then log it.
Tutorialintermediate14 min readUpdated

Why a release can break a path without anyone noticing

A fault that appears only on one browser, device or payment method is the kind the diagnosis of an overnight fall in conversion lists as a bug that only some visitors meet and as checkout or payment failing for some customers.

The advice to walk the paths already appears as a single line in each of three prevention routines. The diagnosis above asks for the journey to be walked, after every release, on the browsers and devices that carry the sales. The diagnosis for a mobile gap asks for a walk from landing page to confirmation over a throttled connection. The diagnosis for abandoned forms asks for every live form to be filled in on a phone and a desktop and the entry confirmed to have arrived. The tracking verification checklist asks for a monthly pass over the key journeys and for a run after a release that changes a form, the checkout or the templates the tags rely on. The measurement governance playbook suggests the same monthly verification and holds back a release that alters a purchase, a lead form or a key event until a second person, not the implementer, has verified it. None of them says what the walk consists of. This page does.

The walk has a narrow job. It does not judge the design, and it does not prove the tracking, which a checklist does in the analytics tools. It asks whether each path that earns money still works for the visitors who carry the conversions, ends in the business's own record, reaches the person who has to act, and is counted once.

CheckWhen it runsWhat it proves
This walkAfter any release that touches a path or what the path sharesThe path works for the devices and methods that carry conversions and ends where the business needs it
Tracking verification checklistAfter tag, pixel, conversion action or banner changes, after a release that changes forms, checkout or shared templates, and monthlyA real action produces the planned event in every system that should receive it
Landing page checklistBefore paid or email traffic reaches a page, and again after a redesign or when the form or its destination changesThe page is ready: message, form, proof, speed and tracking
A/B testing guideBefore a test launchesEach variant renders, assigns and records as the plan says
Evaluating a redesignBefore and after a relaunchWhat to record, which templates to test and which rollback rules to agree, so the new site is judged fairly
Live error response SOPAfter a report that something live is wrongThe error is confirmed, contained, corrected and logged

Before you start

  • The release's change list. What the release touched: templates, forms, checkout steps, scripts, tags, the consent banner, third-party widgets and content such as prices. A developer or agency who can say what changed, and what the changed parts are shared with, saves the guessing.
  • The conversion outcomes and the record that proves each. The order system, the booking system, the account database, and the inbox or customer record system that receives enquiries. The page a visitor sees at the end is not the outcome; the record is.
  • The site's own breakdowns. Conversions by device, browser, source and payment method; the funnel analysis tutorial builds the device and source segments.
  • A convention for test records. The martech guide to test environments and test data covers marked records, invented details and exclusions; agree the marker before the first walk, not during it.
  • Real devices. The phones and browsers the breakdowns point to, and a desktop for comparison.
  • Access to the far end. The order system, the customer record system, the inbox that receives notifications, the mailbox for confirmations and the analytics debug view.
  • A walker who did not build the release, following the script written in the steps below.
  • A record of each walk, with dates, screenshots and the release it belongs to.

The steps

  1. List the paths that earn money and where each ends

    Write one line for each outcome the site exists to produce: an order, an enquiry or quote request, a booking, a sign-up or trial start, a payment on an account. For each, name the entry points that matter, such as an advertisement landing page, an email link, a search result or a direct visit, and name the record where the outcome has to appear. The guides to sign-up flows and booking flows describe what those journeys contain. Keep the list to outcomes the business counts as success. A page that merely informs is not a path, and a short list is a list that gets walked.

  2. Read the release and mark what it touched

    Go through the change list and mark each path as touched, shares something that was touched, or untouched. A touched path is walked in full. A path that shares a template, a script, a tag or the banner with something that changed is walked too, because shared parts are how a change in one place can break another. An untouched path that does not share anything can wait for the scheduled walk. If the change list is vague, treat every path as touched and tighten the list once the developers have named what moved.

  3. Choose the devices, browsers, entry points and payment methods from the site's data

    Open the breakdowns and take the combinations that carry the conversions, add the slowest common phone, add each live payment method, and add the entry points the campaigns currently send traffic through, including a link opened inside a social app's in-app browser, which may not behave like the phone's own browser. Date the list and keep it short enough to run before the next campaign or email send goes out. Revisit it whenever the mix moves, so it does not become a record of last year's visitors.

  4. Prepare marked test records that cannot be mistaken for customers

    Use invented details with the agreed marker, and put the exclusions in place before anything is submitted: the analytics property's internal traffic rules, the report filters, and the audience and upload exclusions. A dedicated test product, a form sent from an address the business controls, or a transaction in the provider's test mode, as the tracking verification checklist suggests, keeps the walk from becoming revenue. Where none of those is possible and a real payment is unavoidable, agree the refund and the marking beforehand. That is general practice, not documentation of any payment provider. Do not use real customers' details.

  5. Write the script for each path

    For each path, write the actions in order and the result expected at each one. Include the ordinary awkward inputs that real people supply: a phone number written with spaces, a long name, an address line that wraps, a value typed in the wrong format that the form should explain rather than refuse. Include the moments the release may have changed: the price and the delivery cost shown before payment, the error message when a required field is left empty, the confirmation page after it is reloaded, and what the back button does. The forms guide and the checkout guide set out how errors, costs and payment should behave, so the script can ask for that behaviour by name. One table across the paths is enough to start with:

    PathStartActions to scriptWhere it must endWhat to count
    PurchaseEach campaign landing page and a direct visitAdd an item, change a quantity, choose delivery, pay with each live methodAn order in the order system with the right items and valueOne purchase event with the order's transaction identifier
    Enquiry or quoteAn advertisement, an email link and the contact pageFill the form with awkward but valid values, leave a required field empty, submitA record where the team works, with its source, and a notification that arrivesOne lead event for the enquiry, even if the confirmation page is reloaded
    BookingThe booking page from a phoneChoose a slot, change it, confirm, cancelA booking in the booking system and a confirmation messageOne booking event
    Sign-upThe sign-up call to action and the pricing pageCreate an account, verify it, reach the first screenAn account that can sign in, in the right planOne sign-up event
  6. Walk every path on every chosen combination

    Start each walk from a clean state, in a private window or a clean browser profile, as the tracking verification checklist asks, and check that the consent banner appears as a new visitor sees it and can be dismissed with a keyboard. Walk the path as the script says, from each chosen entry point, on each chosen device and browser, and note the time each step takes as well as whether it works: a path that completes slowly on a phone over a mobile connection is a finding. The landing page checklist makes the same request, with a mid-range phone and a mobile connection in place of office wi-fi. Lab impressions are for finding causes, and the speed and mobile guide explains why real-user data, which the Web Vitals documentation describes, is the judge of speed. On any path whose forms or controls the release touched, repeat the keyboard-alone pass from the accessibility guide, which draws on the Web Content Accessibility Guidelines.

  7. Follow each result to the record and to the person who must act

    A confirmation on the screen is not the end of the path. Open the order system, the booking system or the customer record system and find the test record with the values that were entered, its source, and its owner or queue. Check the notification and the confirmation message, in the inbox and in the junk folder, because a message that is sent and filed as junk looks, from the visitor's side, like a message that was never sent; the diagnosis of missing order and account emails separates the two. Where the record is a lead, ask whether the person who has to respond can see it: the lead routing QA checklist covers the machinery from form to salesperson, and this step asks only that the lead arrived with its source where the responder works.

  8. Check that each conversion is counted without repeats

    In the analytics debug view, confirm that the outcome's key event is recorded, since events are the unit of measurement, as the Google Analytics documentation on events describes them. When it fires is a rule of the site's own guides: for a form, after the server accepts the submission and not on the click, as the form tracking guide sets out. Reload the confirmation page and use the back and forward buttons to confirm the count does not move; the confirmation pages guide counts each conversion once and explains why the order's transaction identifier, with deduplication, is what stops a repeat. Use the container's preview mode as the debugging tutorial describes. Once a full day has closed, compare that day's conversions in analytics against the business's own records of that day, as the first check in the diagnosis of a sudden rise in the rate does, and split the comparison by the path and the payment method the release touched. Marked test records stay out of that comparison, because their job was the walk. A debug view that looks right does not prove that the report will agree.

  9. Record the walk and decide what a failure triggers

    Write the result by path and by combination: pass or fail, who walked, when, which release, with a screenshot of anything that failed. Log the release in the measurement change log, as that tutorial asks of a release that touches tracked pages or the checkout, and keep the walk record beside it. Then decide what each failure triggers before the next release, not during it. A live page on the path that is wrong for customers is a live error, which the live error response SOP handles. A path that works while its events do not is a measurement fault: the diagnosis of purchases not tracking covers purchases missing for one payment method, and the tracking outage response SOP handles the response. For either, settle beforehand whether the release is reversed at once. The framework on one-way and two-way doors asks how cheaply a decision can be walked back; a two-way door can be acted on, watched on a set date and reversed if the result misses the expectation. A rollback is planned before the change goes live, so the previous version and the person who may choose it are named in advance.

  10. Clean up and set the next walk

    Remove or mark every test order, lead, booking and account, cancel what must be cancelled, issue the agreed refunds and confirm no marked record leaked into a report, an audience list or a platform upload. Then set the next walk: after the next release that touches a path or what it shares, and on the schedule the key journeys need. Update the walk list from the breakdowns, retire combinations that no longer carry conversions and add the ones that now do. Store the script beside the list so that anyone who did not write it can run it.

What a good record looks like

The record is short and comparable from one release to the next. A table with a row for each path and a column for each chosen combination, filled with pass or fail and a reference to the evidence, is enough. Beside the table, keep the list of what was not walked and why, such as a combination retired from the list or a path judged untouched, because the gaps are what a later investigation needs. Compare the record with the release's change list: every touched path should appear, and every failure should appear with its decision.

A weathervane maker walks its paths after a checkout release (illustrative)

Illustrative example — not a real result

Karduxon Weathervanes walks its paths after a checkout release

Karduxon Weathervanes, an invented workshop that makes weathervanes and finials, takes orders for stock pieces through its shop, quote requests for commissioned pieces through a form and applications from trade buyers through a sign-up. On Tuesday evening a release moves the delivery choice into a new component and updates the payment provider's embedded form. Over the last full month the shop took 1,500 orders: 62% on phones, with three browser and device combinations carrying 88% of orders, card payment on 71%, a wallet button on 24% and bank transfer on 5%.

The walk list takes those three combinations plus the slowest common phone, each of the three payment methods, and entry points from the email newsletter and a social post opened inside the social app. The next morning, before the day's newsletter goes out, a colleague who did not build the release walks the purchase, quote and trade paths from a clean state, using the marked test customer and a test-mode payment.

Two faults appear. On one phone browser the delivery step refuses a valid postcode written with a space, with an error that disappears off screen, so the path cannot be completed. And wallet orders reach the order system with the right items, but their purchase event is missing in the debug view, because the wallet no longer returns the buyer to the confirmation page. The quote and trade paths pass, and the quote request arrives in the team's queue with its source. The walker enters the release and both failures in the change log, the owner follows the failure rule agreed before the release, reverting the postcode rule at once and restoring the wallet return page that morning, and the comparison for the first full day after the fix finds that analytics and the order system agree on that day's real orders, wallet orders included. Without the walk, the first sign would have been a reported conversion rate about a quarter lower, and the shop would have been left asking whether the new delivery component had cost it customers.

The funnel and checkout section sets this routine beside the journeys it protects, from the page where a choice is made to the page after the conversion.

Common mistakes

  • Walking only the page that changed. Paths share templates, scripts, tags and the banner, so a change in one place can break another. Walk what the change touches and what it shares.
  • Testing on the builder's own device. The visitor's phone, browser and connection are the test. Take them from the conversion breakdowns, not from what is on the desk.
  • Stopping at the thank-you page. The outcome is the order, the lead or the booking in the business's own record, seen by the person who must act.
  • Using a banner that is already accepted. A warm browser hides what a new visitor meets. Start clean, and check the banner as well as the page.
  • Trusting a debug view alone. An event that fires once in the debug view can still be missing or doubled in the report. Compare a closed day's conversions with the business's records once the day has ended.
  • Letting test records into the numbers. Mark them, exclude them before the walk and remove them after it.
  • Signing off one's own release. The playbook named above asks for a second person to verify a release that touches a purchase, a lead form or a key event, so the builder is not the verifier.
  • Keeping no record. Without one, nobody can say later what was checked, on what, and what was decided about each failure.

Frequently asked questions

How often should conversion paths be walked?

After every release that touches a path, or the templates, scripts, tags and consent banner those paths share, and on a schedule for the key journeys, because sites change between releases too. The measurement governance playbook suggests a monthly verification of the key journeys as a place to start rather than a requirement. The rhythm that matters is the one written down and kept.

Is this the same as a usability test or an expert review?

No. This walk asks a narrower question: does each path still work from the visitor's side to the business's own record, on the devices and with the payment methods that carry the conversions. It needs a script, not a participant, so whoever did not build the release can run it.

Can the walk be done on a staging copy instead of the live site?

A staging copy is the right first place for a change to checkout or forms, provided its tags send nothing to live analytics or live ad accounts; a copy that does is not a test environment. It does not replace the live walk.

Terms used on this page

Frameworks

Sources

  1. [GA4] About events — Google Analytics Help
  2. Web Content Accessibility Guidelines (WCAG) — W3C
  3. Web Vitals — web.dev

Related