Why this matters
Profit targets and a single bidding goal change were the work of the advanced performance marketing level. Here the measure the whole business steers by has gone wrong, and paid media helped: platforms told to buy trial sign-ups find trial-takers, and a north star counting first kits applauds them. Replacing it reaches every report and both platforms' goals; out of order, it leaves months of figures nobody can compare.
The craft club and its two numbers
Grunarkel Craft Club is an invented subscription that posts a children's craft kit each month. Households start with a low-price trial kit and pay full price from the second kit. It wins new subscribers through Google Ads and Meta, which both bid towards trial sign-ups. A paid media manager runs both platforms, with a part-time analyst.
Grunarkel Craft Club: what the team has
| Quarter | First kits shipped a month | Trial households paying at month four | Paying households at quarter end |
|---|---|---|---|
| Spring, last year | 1,800 | 41% | 9,200 |
| Autumn | 2,300 | 33% | 9,400 |
| Spring, this year | 2,900 | 27% | 9,400 |
- Change log: no tracking, consent or definition entry for two years.
- The month-four fall holds on Google, Meta and organic sign-ups, and for both kit age ranges.
- Strict count (no repeat trial-takers or one-off gift trials): 1,700 rising to 2,500 a month; its month-four retention fell from 43% to 30%.
- Inputs: every one met its plan. Trial households paying for a second kit: 52% this spring.
- Proposed north star: settled households, whose third paid kit has shipped and who are still subscribed. The agency prefers first paid kits, which move within a month.
- Subscription rows: every shipment and payment since a billing migration 24 months ago.
- Dashboards reading first kits: 8, of which the agency report and the board sheet compute it themselves.
- Target: £14 per trial sign-up. Running tests: a landing page and a Meta creative test, both on trial sign-ups.
- Busiest season: gifting before Christmas.
Diagnose before you replace
The north star change guide clears four ordinary explanations before naming a fault, and the materials settle each. No log entry falls in the window, so measurement did not move. Retention fell inside every channel and kit range, so the rate changed, not the mix. Counts run into thousands and the fall persists across pooled quarters, so it is no small-number swing. Every cohort read has reached month four, so nothing is still to arrive.
A loose definition would leave the strict count following retention, but it has parted too. A weak input would leave the north star stalled, yet it rose. The fault is the north star itself: a first kit is a low-price trial that does not mean a household will pay full price, and trial bidding lifts the count without adding value.
Task
- Say which line of the materials rules out each ordinary explanation.
- Name the fault and the evidence that separates it from the other two.
- Check settled households, and the agency's alternative, against the choosing criteria.
- Re-base in order: definition, history, dashboards, targets, bidding goals platform by platform, running tests, record.
Worked answer
The replacement. Settled households pass the five tests of the north star and OKRs framework: they count households that kept crafting past the trial; the subscription rows count them by cohort for two years; marketing, kit design and customer service each own an input below them; and trial bidding cannot lift them unless households pay three times. The precedence check from the Leading IndicatorA measure that moves before the business outcome it is expected to predict, paired with a lagging measure that confirms the outcome later. entry runs on the club's own cohorts. First paid kits fail the gaming test, since a second-kit discount would lift them, but they earn a job below.
Definition, history and dashboards. Settled households enter the metrics layer under their own name, and the metric's owner runs the metric definition change checklist with finance. History is restated back to the billing migration, and older charts mark the break. Filtering the inventory by metric key finds all eight dashboards; the two that compute first kits themselves are rebuilt.
Targets. The trial target lapses with trial bidding. Cost per first paid kit is set from closed figures and paired with a guardrail metricA metric an experiment must not worsen beyond an agreed bound, watched beside the primary metric that decides the test.: the share of first paid kits bought at a discount.
Bidding. Settled households arrive months after the click, so the conversion events guide points to an earlier outcome sent back from the subscription system. The first paid kit becomes the optimisation event, while both platforms are judged on settled households. It is first recorded on both platforms without steering bids, as a secondary conversion actionA Google Ads definition of one action to count, such as a purchase or lead, with its source, category, counting method, value and conversion window. on Google, and compared with subscription rows. The conversion goal change checklist stages it: Google, then Meta, outside the gifting season, with targets and budgets held in each switch week.
Running tests and the record. Both tests finish on trial sign-ups, filed with a note that their metric no longer feeds the north star. The switch enters the measurement change log, every reader is told what changed and from when, and a decision logA running record, one row per decision, of what changed and why, what was expected, who decided and when the result will be checked. entry holds the evidence and a date to check the new measure against revenue.
Grunarkel Craft Club: the arithmetic and the order
Households reaching month four: 1,800 × 0.41 ≈ 738 a month then, 2,900 × 0.27 ≈ 783 now: trials up 61% for 6% more households that stayed.
Precedence check, the 12 monthly cohorts that have reached month twelve: the half with more households reaching a third paid kit earned £71 per trial household by month twelve, the rest £49.
Starting reference: £14 per trial ÷ 0.52 first paid kits per trial ≈ £27 per first paid kit, replaced by the closed figure after the window.
| Month | Step |
|---|---|
| February | Definition signed, history restated, dashboards rebuilt; first paid kit recorded on both platforms |
| March | Google compared with subscription rows, then switched; targets held that week |
| April | Meta switched in one edit; Google target reset from closed figures |
| May | Meta target reset; both tests closed on their own rules |
Common mistakes
- Replacing before diagnosing. A loose definition or a weak input needs a smaller repair.
- Bidding on the north star itself. It lands months too late to steer an auction.
- Moving live tests to the new measure. They finish on the rules they started with.
- Resetting targets before the window closes. The count is incomplete.
