Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Analytics › Expert

Handling a Tracking Outage and a Wrong Report for a Stationery Brand

You run an invented stationery brand's tracking outage after the fact: a theme update stopped its purchase event during a product launch, data came back by accident, and the monthly board report built on the gap led to a budget cut before anyone noticed. You finish with the outage confirmed, traced, fixed and annotated, the wrong figure sized and restated, and the people who acted on it told in the right order.
ExerciseExpert levelModule 3: Measurement as a governed service5 minUpdated

Why this matters

The advanced analytics level wrote metric definitions and a measurement strategy. Measurement run as a service is tested when it fails quietly and a decision has already been taken on the result. The tracking outage response SOP runs the incident; the tutorial on correcting a number after a report has gone out runs the correction.

The stationery brand and the missing week

Fulmaquin Stationery is an invented online stationery brand; subscription box sign-ups use a hosted page with their own event. The ecommerce manager owns analytics, a theme agency the site's code, the marketing manager Google Ads and a freelancer paid social, and the managing director presents a monthly report to the board.

Illustrative example — not a real result

Fulmaquin Stationery: the launch week and the board report

Launch weekShop ordersAnalytics purchasesRelease notes
Monday1646110:15 theme update: planner range and a new confirmation template
Tuesday1712
Wednesday1580
Thursday1663
Friday1491
Saturday1377012:40 basket hotfix restores the previous basket and confirmation templates
Sunday128121
Paid social, last click in analyticsAugustSeptember as sent
Spend£5,400£7,800
Purchases360260
Cost per order£15.00£30.00
  • September paid social: £4,300 and 37 purchases from Monday to Saturday of launch week, £700 and 36 on the Sunday, £2,800 and 187 on the other 23 days. Paid search as sent: £13.50 per order against £11.90 in August; its measured days held £4,850 and 428 purchases.
  • Shop orders: 3,980 in August, 4,150 in September. Meta, which also receives every order from the shop's server integration, reported 471 and 655 purchases.
  • The board cut October paid social from £7,000 to £3,500, finance reforecast, and the freelancer was told to halve budgets from Friday. On Thursday he replied that Meta showed purchases holding up through launch week. The tag container had no new version that month; the November release reapplies the new templates.

Incident first, then the correction

The correction tutorial separates data never collected, which the SOP annotates, from figures computed wrongly on their way into a report. This case is both: an outage, and a figure built on it that the board acted on. The SOP comes first, because the correction is sized from its window and cause. Telling people waits only as long as confirming and sizing take, because the decision is still being carried out.

Task

  1. Confirm the outage, record its window and how it was found, and write the check and the route that would have found it on the first morning.
  2. Triage the cause, and say whether Saturday's recovery means it is fixed.
  3. Assess the impact on bidding and reports, and what paid media does now the window has closed.
  4. Fix, verify and annotate.
  5. Size the wrong figures for every period and segment the report showed.
  6. Restate the report and write the correction note.
  7. Decide who hears, in what order, and what is recorded.

Worked answer

Confirmation and detection. The ecommerce manager compares analytics purchases with shop orders, the source of truth, day by day. On the Sunday they run just under orders, as usual; from the Monday release to the Saturday hotfix they all but vanish. That is the window, with evidence at both ends, and she declares the incident. No check found it; a remark about a budget cut did. The remedy is the comparison of key events with orders that the data quality guide sets, against the tolerance written in the measurement plan. Run it the morning after every theme release as well as in its weekly routine, as the guide advises for releases that touch tracked templates; the SOP asks that a recurring cause, like a theme release with no tracking check, becomes a change to the release process. A gap outside the tolerance triggers the SOP, and an Escalation Route to the ecommerce manager, the marketing manager deputising, serves anyone who suspects a break.

Triage. With no new GTM version, the fault sits in the page: the new confirmation template lacks the purchase push. The hotfix restored the old template by accident, so the cause is still there and the November release brings it back.

Impact and protection. Analytics and Google Ads lost purchases for the window; Meta's server events kept arriving, so its bidding saw the sales. With the window closed, the marketing manager makes no reactive changes, applies a data exclusion only if Google documents one for that strategy and window, and uploads the window's orders where the platform supports it. The September report, the dashboard and every comparison with September carry the gap.

Fix, verify, annotate. The agency adds the push to the new template before November, tested in preview, and the tracking verification checklist joins every theme release. Real purchases against orders verify it after release. The window is annotated in analytics and dashboards and entered in the measurement change log.

Sizing. Unrecorded purchases cannot be recreated, so no full-month analytics figure exists. Read each channel on the days measurement worked, as below; conversion rate and comparisons with September moved too. Do not substitute Meta's purchases, which use another attribution, or present an estimate as measured.

Restatement and note. The dashboard and archive carry a corrected September with the window shown as a gap; the sent version stays, marked superseded. The note opens on the restated cost per order and the decision it bears on, then the cause and the fix.

Who hears. Confirming and sizing take hours, so the managing director hears that day, in person, before Friday's cut; whether it stands is the director's call with the board. The board, which made the cut, hears from the director before the general note. Finance and the freelancer are told whether the cut holds. The correction note then goes to the distribution list. One dated Change Log entry links the incident and the restated report, and the monthly pack gains a reconciliation of purchases against orders before it is sent.

Illustrative example — not a real result

Fulmaquin Stationery: the restated figures

Measured days onlySpendPurchasesCost per order
Paid social, 23 days plus the launch Sunday£3,500223£15.70
Paid search, the same days£4,850428£11.33

Against £15.00 and £11.90 in August, neither channel's cost per order doubled. The director held the cut for a week, then restored £3,000 of it.

Common mistakes

  • Calling it fixed because the data came back. The cause returns with the next release.
  • Sending the general note before the director and the board have heard. The people who acted on the figure hear first.
  • Patching September with Meta's figures. Two attribution methods in one number.
  • Overwriting the sent report. Readers cannot see what changed.
  • Reversing the cut on the analyst's authority. The correction informs the decision; it does not make it.

The marketing measurement audit, used in the capstone, would mark this incident down on its data quality and change log rows.

Pages this lesson applies

Terms used on this page

Also related

Related