What this checkpoint covers
Suskeird Tiles, an invented online retailer of floor and wall tiles, steers by sample orders, and Google Ads and Meta both bid towards them. Tile sales have not followed.
Suskeird Tiles: the position
| Measure | A year ago | This month |
|---|---|---|
| Sample orders a month | 3,100 | 4,300 |
| Tile orders a month | 640 | 650 |
| Tile orders per strict sample (one per tile per household, promotional free samples excluded) | 31% | 30% |
| Tile orders per sample, as counted now | 21% | 15% |
- Proposals: bid towards tile orders that follow a sample; count samples strictly; report revenue net of unopened boxes returned; replace the north star.
- The board pack and the agency's contract quote revenue as now counted until the year end.
- Dashboard inventory: 12 dashboards carry the sample orders key; the analyst knows of 3.
- The agency's fee rises when cost per sample falls below £4.
- This quarter's key results: sample orders up to 4,800 a month; tile orders per sample up to 25%.
- Running for three weeks: a free next-day sample delivery test, primary metric sample orders per visitor.
Task
- Decide an action and its reason for each scenario first; a right action on a wrong reason scores nothing.
- Reread the linked page behind any miss.
Questions
- Finance wants revenue net of returned unopened boxes from next month. New version, or a new name?
- The paid media lead wants tile orders after a sample as the Google primary conversion on Monday. It has not been recorded there yet. What comes first?
- The analyst plans to switch the three dashboards he knows to the new sample definition. What is missing?
- The paid media team, judged on cost per sample, offers to write the tighter definition and a target for it. Your response?
- The founder reads the table as a failed north star. Is it?
- Suppose the strict count had parted from tile orders too. The agency proposes room planner sessions as the new north star, because they doubled within weeks of the planner's launch. What do you ask for?
- What happens to this quarter's key results if the north star does change?
- The delivery test's owner wants it re-read on the new sample definition. Do you agree?
Answers explained
- A new name. Old and new figures will both be quoted until the year end, which is the case the versioning rules give a separate name; otherwise a new version would do. Either way the old measure is marked deprecated with a date to be withdrawn, rather than edited in place. See versioning and change.
- Observe it first. Add it as a secondary action, compare its count with the order system for a closed period, and check it fires and counts once before it carries bidding. On Meta the same applies before the optimisation eventThe conversion event an ad set or campaign is told to optimise delivery towards, such as a purchase, lead or install. moves. See the before switching group.
- The other nine, and the contract. Filtering the inventory by the metric key lists every dashboard, including any that calculate the figure in their own fields. The agency's fee rests on the old definition, so the agreement is rewritten too. See everything that reads the metric.
- Not theirs alone to decide. The owner approves with sign-off from outside the team judged on the number, since a team that redefines its own target can meet it without improving anything. Any new target carries a guardrail metricA metric an experiment must not worsen beyond an agreed bound, watched beside the primary metric that decides the test., such as tile orders per sample. See when a measure becomes a target.
- No. The strict count still follows tile orders while the loose count has drifted away, which marks a loose definition. The repair is a tighter version of sample orders, not a new north star. See telling the three faults apart.
- Evidence that it comes first. A measure that moves fast is not shown to lead sales. Run the precedence check on Suskeird's own cohorts: did cohorts with more planner use go on to more tile orders? Record the result first. See the leading indicatorA measure that moves before the business outcome it is expected to predict, paired with a lagging measure that confirms the outcome later. entry.
- Each one is reviewed openly. The first, tied to the old north star, is rewritten with a note of the change or visibly dropped; the second stays if it still feeds the new one. Sample orders remain a KPIA metric tied directly to an objective. that is watched, not targeted. See the mid-cycle check.
- No. The test finishes on the metric and decision rule written before it began, and is filed with a note if that metric no longer feeds the north star. See writing the decision rule first.
Answer five: the arithmetic
Tile orders per sample as counted: 640 ÷ 3,100 ≈ 21% then, 650 ÷ 4,300 ≈ 15% now. Strict samples: 640 ÷ 0.31 ≈ 2,060 then, 650 ÷ 0.30 ≈ 2,170 now. Of the 1,200 extra samples, about 110 are strict; the other 1,090 are repeat samples of a tile or promotional free ones, which is why the loose rate fell while the strict rate held.
Where to go next
Missed the fifth or sixth question? Redo the craft kit subscription exercise, which reaches a different verdict. For the signals beneath these changes, see the advanced performance marketing level.
