When to run this checklist
Run it once a change to a reported metric has been decided and while no report or dashboard yet uses the new version: fees added to cost per lead, refunds and cancellations taken out of revenue, a different date a conversion is counted on, a new attribution rule or channel grouping, a new source of truthThe system whose figure is treated as authoritative for a given number, such as billing for revenue or the CRM for pipeline, with other tools reconciled to it. for a figure, a qualified lead or a lifecycle stage redefined, or a north star or other headline measure replaced. Run it again for every later change to the same metric. The comparing periods guide counts definition, attribution and channel grouping changes among the breaks in the data, and the tutorial on automating lifecycle stage changes makes the same point about stages: every report that reads them moves on the day the new rules go live.
The person who ticks it is the metric's owner, with the analyst who maintains the definitions and someone from each team that reads or is judged on the number: finance for a measure it also reports, sales for a shared lead definition. The method sits in a handful of pages. The metrics layer guide writes the definition and its versioning rules; dashboard governance keeps the inventory that finds every dashboard a change touches; the measurement change log tutorial records it; and the measurement governance playbook names who may sign off a re-baseline. Two guides work through the whole change for a particular kind of metric: the guide to changing a north star metric re-bases everything that hangs from a headline measure, and the tutorial on moving from MQLs to account qualification changes a definition two teams share.
Neighbouring lists take the cases this one does not. A change to the conversions an ad platform optimises for, their values, their counting or the attribution windows is gated by the conversion goal change checklist, because those settings change what the platform learns from as well as what it reports. Each dashboard that reads the changed metric goes through the dashboard launch QA checklist before it is shared again. A redefined qualified lead changes routing as well as reporting, and the lead routing QA checklist tests the routes. A new tag or event that changes what is collected is proved by tracking verification first. And a number that was simply wrong is not a new definition at all: it is an error, which the tutorial on correcting a number after a report has gone out puts right in the open.
Before you start
- The current definition, from the metrics layer or a dictionary laid out like the KPI dictionary, with its owner, version and status.
- The proposed definition written in the same fields: formula, grainWhat one row of a table represents, such as one order, one order line, one session or one customer per day; queries and metric definitions must match it., source tables, baked-in filters, attribution rule, time rule, owner and known differences.
- The reason for the change and who asked for it, in writing.
- The dashboard inventory, with every dashboard's metrics listed by their keys.
- The targets, key results and agreements that quote the metric, including any shared with sales, agencies or partners.
- The running tests, each with the primary metric and decision rule written before it started.
- The raw rows a restatement would need, and how far back they go.
- The measurement change log and the next scheduled report.
The checklist
The groups follow the order of the work: make sure it really is a definition change, settle who agrees it, decide what happens to history, find everything that reads the metric, move the targets, tests and models that hang from it, then record and announce the change. A line in the first two groups that cannot be ticked halts the change, the targets group is settled before the switch date, and the last group runs past it and closes at the quarterly review.
Is it a definition change
Owners and agreement
History and the break
Everything that reads the metric
Targets, tests and models
Record, announce and review
Where each check is explained
The widget keeps the lines short so they can be ticked; the reasoning lives in the guides. Each check links to the page that explains it.
If an item fails
The first group decides whether this list applies at all. If the number was calculated, joined or filtered wrongly, stop and follow the correction tutorial, which restates the figure as an error and tells the people who acted on it. If the request comes from a number that moved and nobody has yet explained why, run the tracking, calendar and volume checks and the mix and rate split in the guide to explaining a change in a marketing metric first: a tag that stopped firing, a short month or a KPIA metric tied directly to an objective. built on a handful of conversions can make a sound definition look broken.
The agreement group is a hard stop. A change with no named owner becomes whatever the last person to edit a dashboard decided, and a change approved only by the team whose target reads the number is the pattern the Goodhart's law framework warns about: a measure people are rewarded for moving can be moved without anything improving. The qualification tutorial writes that rule into its agreement with sales, so any change to a shared criterion goes through a recorded decision that both teams sign. Where finance also reports the measure, the diagnosis of finance not trusting marketing numbers gives the prevention: definitions move only when both teams agree, and each change carries its date and a note in the following report. A headline metric waits for the marketing lead, as the governance playbook's re-baseline gate requires.
The history group decides how the change is read. Where the raw rows do not allow a restatement, the comparing periods guide offers two honest alternatives: compare only within one side of the break, or switch to a measure the change did not touch. A side-by-side run needs an end date agreed at the start; the qualification tutorial fixes one at the outset so the run closes with a decision, and points out that the change can be undone only for as long as the old stage and its reports are kept running, which is the point of a parallel runRunning the old and new versions of a process, platform or tag set side by side for an agreed period, compared on one measure, before the old one is retired..
The readers group can go ahead with a note, provided every item is listed with an owner. The metrics layer guide treats a dashboard that calculates a defined metric for itself as a defect, and the dashboard governance guide removes its certification until it reads from the layer again; a dashboard already flagged for retirement can start its sunset instead of being rebuilt. A reconciliation test left on the old rule will fail every day, and the data tests tutorial warns that a check which fails daily for that reason trains people to stop reading its alerts, the important ones included. The academy checkpoint on redefining metrics and replacing a north star tests these groups through scenarios at a tile retailer, from choosing a new version or a new name to the forgotten dashboards and agency fee that read the old count, and a team offering to write the definition it is judged on.
The targets group has to be settled before the switch date, or a team appears to improve or decline on a day when nothing changed. Where the metric carries key results, the north star guide reports re:Work's guide to OKRs, as the north star and OKRs framework cites it, asking for key results that are measurable and can be checked; by that framework's own rule, each rewritten key result states its data source and the reading it starts from, its BaselineAn estimate of what a metric would have done without a given change, campaign or promotion, against which the effect is measured.. Pair a new target with a guardrail metricA metric an experiment must not worsen beyond an agreed bound, watched beside the primary metric that decides the test. and rework the branches beneath the measure with the KPI tree framework. Where a target moves to a different measure, the qualification tutorial applies Kaushik's measurement model, which derives every measure from the business objective, to set marketing's target on accepted accounts rather than a count of handed-over accounts. Tests already running keep the rule written before they started, because switching a live test to the new definition lets the team pick whichever reading flatters the result. The academy exercise on re-basing paid media when a craft kit subscription changes its north star practises these groups for a north star change that also re-bases bidding, with history restated back to a billing migration, every dashboard found by its key and a trial sign-up target replaced by one paired with a guardrail.
In the last group, a change that is not logged, noted and announced is a silent re-baseline, which the governance playbook warns makes every comparison across the break wrong. Write the change logA dated record of changes made to tracking, accounts, reports, lists or rules, with who made each and why, so later movements in the figures can be traced. entry, carry the note into the reports as the reporting cadence playbook does, and write the decision logA running record, one row per decision, of what changed and why, what was expected, who decided and when the result will be checked. entry with a date to check the change held.
An outdoor gear retailer redefines revenue (illustrative)
Revenue after returns: eleven dashboards checked, one test left alone
Kibbertane Outfitters, an invented online retailer of walking and camping gear, has reported revenue in its marketing dashboards as the value of orders placed, while finance's monthly accounts take out refunds and cancelled orders against the date of the original order. After a quarter in which marketing reported £412,000 and finance £371,000, the finance controller asks for one definition. The marketing lead owns the metric and runs this list with the analyst, the controller and the head of ecommerce.
The first group settles the kind of change. Nothing was calculated wrongly: marketing's figure followed its own written rule, and the order system shows the same refunds, so this is a new version, not a correction or a tracking fault. Written side by side, the old and new entries differ in two fields: the baked-in filter now removes refunds and cancellations, and the time rule counts each refund against the original order's date, so every month stays provisional until the returns period on its orders has closed. So that the board pack and the paid social agency's monthly report, which quoted the old figure, can carry a footnote instead of being reissued, it stays available for one quarter under the name "order value before returns", marked deprecated with its end date.
The marketing lead signs the change as its owner and, since revenue is a headline metric, as the re-base sign-off, and the controller signs for finance; the ecommerce team, whose targets read revenue, is consulted but does not approve it. The analyst restates 24 months from the order system's refund records and publishes a before-and-after: the last quarter moves from £412,000 to £371,000, and the year-on-year trend barely moves because returns ran at a similar share in both years.
The inventory catches what the team would have missed. Filtered by the revenue key, it lists 11 dashboards: 9 read the modelled table and switch with the version, but 2, built by the agency, compute revenue in their own calculated fields and would have gone on showing the old figure. Both lose certification until they are rebuilt on the layer, and all 11 pass dashboard launch QA again before they are shared. The reconciliation test against the shop platform's gross sales report would have failed every day; it now compares like with like. The ecommerce team's revenue targets and the agency's revenue target are re-based on the restated history, a free delivery threshold test already running finishes on the metric written before it began, and the forecast's revenue row is rebuilt from the restated months.
The change enters the measurement change log for 1 October with the reason and the analyst's name, and the October report carries a note beside every revenue figure. The decision log entry sets a check for the January close: do finance and marketing quote the same revenue for the quarter? They do, and the old name is retired.
Common mistakes
- Editing the definition in place. A silent change moves every historical number, and the next reconciliation meeting spends its time finding out why.
- Updating the dashboards someone remembers. The forgotten ones keep the previous figure; the inventory, filtered by the metric key, lists every dashboard the change touches.
- Calling an error a new version. A wrong number is corrected and restated openly; versioning it hides the mistake from the people who acted on it.
- Letting the team judged on the number redefine it. The target can be met without anything improving.
- Switching live tests to the new definition. A test is called on the decision rule it began with, never on a later one.
- Re-basing the chart and nothing else. Targets, key results, agreements, models and reports that hang from the metric move with it.
- Restating without saying so. A restated history that nobody announced looks like a report that changed its mind; publish the before-and-after and mark the date.
The checklists library holds the lists that sit beside this one, from dashboard launch QA to conversion goal changes.
