First, compare members with non-members over the same period, not members before and after
Start with the comparison the programme's own report never makes. The loyalty programme guide reports work associated with the Ehrenberg-Bass Institute arguing that loyalty is largely a by-product of a brand's size and of how easy it is to buy, and that the people a programme draws in are mostly heavy buyers the brand already had. That is why members buying more than non-members proves nothing: members were different before they joined. The repeat purchase and loyalty guide sets the honest reads in their place: members against non-members matched on their behaviour before joining, or the same customers before and after joining, never members against everyone else.
Take the members who joined in a window a year ago. For each, find non-members who bought at a similar frequency and value in the months before that window, and follow both groups over the same period since. If members' frequency rose while matched non-members' did not, something changed, and the next question is which reward changed it. If both groups moved the same way, the programme recorded loyalty rather than causing it. A held-out group is cleaner still: the repeat purchase guide calls keeping eligible customers out of a reward the only clean measure of incrementality, and the third row of the table below covers it.
Then place the brand's figures in context before judging any target. The double jeopardy pattern holds that loyalty measures largely follow a brand's size, so a modest repeat purchase rateShare of customers who placed more than one order in the period. may be roughly what a brand of that size would be expected to have, and a programme that set out to match a larger rival's loyalty was set an unreachable target. Two further readings narrow the table below. Count where the points went: if most were issued on the lines customers already bought every week, the first row is likely. And count who redeemed: if few members have ever reached a reward, the fifth row is likely, whatever the reward design says on paper.
Diagnosis table
Diagnosis
| Symptom | Check | Likely cause | Fix |
|---|---|---|---|
| Members buy often, but matched non-members buy just as often, and most points were issued on routine lines | Members' frequency before joining against matched non-members over the same period; the share of rewards issued on habitual purchases | Rewards going to habitual purchases that would have happened anyway | Move rewards to frequency where customers have a choice, a first purchase in a new category and actions with value beyond the purchase |
| Retention kept falling after launch, and exits follow a late delivery, a stock gap or a poor support contact | Exit reasons and support contacts for the customers who lapsed since the programme began | A programme launched to fix churn caused by the experience, which rewards cannot repair | Fix the reason customers leave first; keep the programme only on top of an experience that already gives a reason to stay |
| Every report compares members with everyone else and the uplift never survives a question | Whether any eligible customers were ever invited at random or held out of a reward | No comparison group, so every purchase by a member looks like the programme's success | Hold a random part of eligible customers out of one reward or tier benefit and read incremental margin against them |
| Most points are earned on one consumable that customers reorder on a steady cycle | Reorder gaps for the most rewarded lines, and whether they cluster | The wrong model for a consumable bought on a steady cycle | Offer a subscription on that line and keep rewards for the range bought on the customer's own schedule |
| Few members have ever redeemed, and most balances sit below the first reward | Redemption share, the spread of balances against the first reward, and whether any message shows the balance | Members who cannot see their balance or reach a reward, so the scheme changes no behaviour | A first reward reachable early, the balance and distance to the next reward in triggered messages, and expiry warnings |
| The overall retention rate is flat while the programme's own report shows improvement | Retention and repeat purchase by acquisition cohort, members beside non-members, at the same age | Loyalty read on a blended rate that cannot show whether members' cohorts retain better | Define retention and repeat purchase in writing and read them by cohort at fixed ages |
Fix, verify and prevent each cause
Rewards go to purchases that would have happened anyway
The loyalty guide asks that rewards go only to behaviour the business wants more of and would not get without them, and the first thing it warns against rewarding is the habitual weekly shop, where rewards mostly return margin to people who were not going anywhere. A programme that issues most of its points on those purchases is the commonest reason a scheme costs more than it returns, and it looks successful in every report that counts members' spending. The double jeopardy page makes the same point from the other side: programmes that reward purchase frequency tend to reward the heavy buyers the brand already has.
Fix: read where the points went by line and by customer, then move the rewards to what the loyalty guide says is worth rewarding: frequency in categories where customers genuinely shop around, a first purchase in a new category or through a new channel, and actions with value beyond the purchase such as reviews, referrals and completed profiles. Where a reward paid as money off has taught customers to wait for it, the diagnosis for customers who only buy in the sale covers that habit; this page covers rewards that change nothing.
Verify: after the change, repeat the matched comparison for the behaviour each reward now targets, such as second-category purchases among members against matched non-members, and read the share of points issued on routine lines, which should fall.
Prevent: before any reward is added, write down the behaviour it is meant to change and how that behaviour will be read against customers who did not receive it, and refuse rewards that cannot answer both questions.
The programme was launched to fix an experience customers leave because of
The customer retention strategy guide answers the question directly: a retention strategy should usually not start with a loyalty programme, because a programme rewards an experience and cannot repair one. Its diagnosis table lists the reasons customers leave, from never reaching first value to a bad experience and a competitor's offer, and most of them are untouched by points. The loyalty guide lists the same sign among the signs a programme is the wrong answer: where customers leave after a late delivery, a poor support contact or a disappointing product, rewards only add cost, because the reason to leave is still there.
Fix: run the retention guide's diagnosis for the customers who lapsed since the programme launched, reading their behaviour, what they said in exit surveys and support contacts, and what the people who talk to customers every day already know; then pull the lever the diagnosis names, whether onboarding, habit, value or service, before spending more on rewards. The foundations hub places loyalty after journey, onboarding and retention for this reason.
Verify: the exit reasons that dominated before the fix shrink in the next quarter's survey and contact tags, and the lapse rate among new customers falls whether or not they joined the programme.
Prevent: keep the retention plan's diagnosis current and read it before every programme decision, and write into the plan what the business will not do, such as adding rewards while a known reason for leaving is unfixed.
There is no comparison group
The loyalty guide says plainly that measuring what a programme changes requires a comparison group, which is why a pilot beats a full launch, and it lists the missing group among its common mistakes: without one, every purchase by a member looks like the programme's success. The loyalty programme pilot test designs the comparison for a programme not yet launched: eligible customers assigned at random, once, to an invitation or a held-out group, with every assigned customer counted whether they joined or not, and the two groups compared on the extra contribution margin per eligible customer once redeemed rewards, rewards still owed and running costs are taken off. For a programme already offered to everyone, the same page gives the alternative: hold a random part of eligible customers out of one reward or one tier benefit and compare.
Fix: pick the reward the business most suspects, hold a random part of eligible members out of it for a period long enough for several purchase cycles and for rewards to be earned and redeemed, keep every other message, price and offer identical for both groups, and value rewards still owed in full rather than at the BreakageThe share of issued loyalty rewards that customers never redeem, because points expire, balances are too small to use or members forget. the business hopes for. The break-even calculator frames the reading: how much extra margin the reward has to create to cover what it gives away.
Verify: the difference between the groups, read with its interval rather than as a single figure, and a follow-on period that shows whether the rewarded group merely brought purchases forward.
Prevent: write the groups, the expected result and the guardrails down before each new reward launches, and keep the held-out customers off every public page and general newsletter that would invite them by accident.
The model is wrong for a consumable bought on a steady cycle
The comparison of loyalty programmes and subscribe-and-save chooses by how customers already buy. Where one consumable is reordered on a steady, predictable cycle, a loyalty programme rewards each reorder after it happens, where a standing order would have secured it in advance and removed the moment at which customers lapse or switch; where customers buy across a range on their own schedule, there is no cycle to agree and a reward can still target frequency or a new category. A programme whose points are mostly earned on one consumable is paying for orders that a subscription would have made the default.
Fix: test the line the comparison's way, by looking at customers who bought it more than once, how long they waited and how much that gap varies; where the gaps cluster, offer a subscription on that line, keep subscription deliveries outside the points scheme so the same order is not rewarded twice, and let the programme apply to the range.
Verify: subscribers read by cohort against similar customers left on reminders, and members' behaviour on the range read against a held-out group, measured separately and on the same basis.
Prevent: decide the boundary between the two schemes in writing, state in both sets of terms whether deliveries earn rewards, and never run a loyalty promotion that offers the subscribed product cheaper than the subscription.
Members cannot see their balance or reach a reward
The repeat purchase and loyalty guide is specific about the two details that decide whether points change anything: points that take a year to become useful do not change behaviour this month, and customers who can see that a reward is close act differently from those who would have to log in to learn where they stand. It reads too much breakage as the programme not working, because members who do not redeem are not being motivated by the rewards, and it names the levers that reduce it: expiry rules, reminder messages and low redemption thresholds. The loyalty guide adds that a member should not need to log in to know how many points they hold, what those points are worth and how far away the next reward is.
Fix: set a first reward reachable within the first few orders, show the balance and the distance to the next reward in the post-purchase and milestone messages the loyalty guide describes, from welcome and balance reminders to expiry warnings, and keep the recognition in those messages free of money off, as the guide to email for loyalty and advocacy sets out. Any change to expiry or balances needs plain terms: the loyalty guide notes that under UK consumer protection law, enforced by the Competition and Markets Authority, a scheme's terms have to be fair and transparent and its marketing must not mislead, which it gives as general direction rather than legal advice.
Verify: the share of members who redeem at least once rises among the cohort that joined after the change, and the behaviour change among redeeming members, not the saving from those who never engage, is what the programme is judged on.
Prevent: report redemption and the spread of balances every quarter beside the headline figures, and treat a growing share of balances stuck below the first reward as a design fault to fix rather than a saving to celebrate.
Loyalty is read on a blended rate
The churn and retention metrics guide explains why a blended retention rate can stay flat while a programme does or does not work: the rate is dominated by long-standing customers, who are usually loyal, so it hides what happens to newer groups until they become a large share of the base. A programme read on that rate cannot show whether members' cohorts retain better than non-members' of the same age, and a report that shows members' blended rate rising may be showing only that the heaviest buyers joined. The repeat purchase guide adds the cohort reading from the programme's side: a scheme that improves retention for customers acquired this year while last year's lapse at the old rate is working on new customers only.
Fix: write down the definitions of active, lapsed and repeat purchase as the metrics guide asks, with a window set from the business's own gaps between purchases, then read retention and repeat purchase by acquisition CohortA group of customers who share a starting event and period, tracked together over time., members beside non-members at the same age, so the curves can be compared. Feed the result into LTVTotal value a customer brings over time. as the loyalty guide does, because a change in frequency that holds across cohorts is what decides how much a programme can be worth.
Verify: the cohort tables show members' and non-members' curves separating after joining, or not, and the blended figure is no longer the headline in any programme report.
Prevent: fix the definitions in a shared metrics layer, put the cohort read on the retention dashboard rather than in the programme team's own report, and review it on the same cycle as the rest of the retention plan.
Prevention: a comparison group from the start and a yearly read of incremental margin against reward cost
Every cause above survived because the programme was never asked the one question that matters: what did it change that would not have happened anyway? Build the answer into the programme's design rather than its defence. Before a new reward, tier or model launches, a random part of eligible customers is held out of it, the behaviour it targets is named, and the reading date is set after several purchase cycles and after rewards can be earned and claimed. The pilot test page gives the shape; the habit is to apply it to every change, not only to the launch. A programme that has never had a held-out group can start now, one reward at a time, and the first result usually settles an argument that has run for a year.
Then read the economics once a year on the loyalty guide's terms. On those terms, the programme earns its keep only when the extra margin it creates is larger than the rewards actually claimed and the cost of running it, a cost that includes the technology, the staff, the messages and the margin handed back to customers who would have bought anyway, which most business cases leave out. Put the comparison group's result beside rewards redeemed, rewards still owed valued in full, running costs and the break-even calculator's view of how much extra behaviour the scheme has to create. Report the programme's return on incremental margin only, and watch the liability of unredeemed rewards as a balance-sheet item rather than a marketing figure. Finance should own that line, as the loyalty guide asks, because a programme treated as a marketing project is one whose costs nobody adds up.
Keep the retention diagnosis in front of the programme. Each quarter, the retention plan's reasons for leaving are re-read from behaviour, exit surveys and support contacts, and the programme is judged against them: rewards cannot fix a stock gap, a late delivery or an onboarding that never reaches first value, and a scheme that keeps paying while one of those remains unfixed is paying for the wrong problem. The customer retention audit scorecard rates measurement, experience, service, exit and loyalty together, and it is the better place to start when several of these habits are missing at once, because it ranks the weaknesses that lose the most customers before any new reward is launched.
Finally, make the programme's own figures honest by default. Redemption, the spread of balances against the first reward, the share of points issued on habitual lines and the cohort curves of members beside non-members belong in the quarterly report as standing items, so the signs of a scheme that changes nothing appear within a quarter rather than after a year. The troubleshooting collection for customer experience lists the neighbouring symptoms that often arrive with this one: a churn rate that keeps rising is the reason most programmes were launched, and support volume that keeps rising is often the experience fault the rewards were meant to paper over. A programme that is read this way can still disappoint, but it disappoints in its first year, with its balances honoured and its terms plain, rather than in its fourth.
A pet shop chain finds its points went to regulars (illustrative)
Moravenne Pet Stores
Moravenne Pet Stores, an invented chain of 14 pet shops, launched a points scheme 13 months ago to slow a fall in repeat purchases. The programme has 31,000 members, and the quarterly report shows members visiting 7.8 times a quarter against 4.1 for non-members, which the board has read as proof that the scheme works. Repeat purchase across the whole base has nevertheless fallen for 5 consecutive quarters.
The marketing lead matches 6,000 members who joined in the first two months to non-members who bought at a similar frequency and value in the 6 months before launch. The matched members visited 7.6 times a quarter before joining and 7.8 since; the matched non-members moved from 7.5 to 7.7 over the same period. The scheme recruited the chain's most regular buyers and changed their frequency by almost nothing. Of the £188,000 in points issued in the first year, 71% was earned on dry food and cat litter, the lines those customers bought every week. Rewards redeemed cost £118,000, running costs £36,000, and the matched comparison puts incremental margin at roughly £22,000.
The lead then reads the exit survey the team had stopped opening: 44% of lapsed customers cite items that were out of stock on more than one visit, and support contacts tagged to stock gaps rose through the same period the scheme was running. Stock availability goes to operations as the first fix. The programme is redesigned around two behaviours the matched read showed moving nowhere: bonus points on a first purchase in grooming, toys or accessories, and on visits before midday on weekdays, when the shops have spare capacity. Points on routine food lines stop. The redesign runs as a pilot: 5,000 eligible members are held out of the new rewards for 6 months, with the reading date set after rewards can be claimed, and the board's next report compares the two groups on margin after reward cost rather than members with everyone else.
Common mistakes
- Reading members against everyone else. The customers who joined were already the best customers; compare like with like or hold out.
- Rewarding the weekly shop. Most of the cost goes to purchases that would have happened anyway, with little change in behaviour.
- Launching points to fix churn. A programme rewards an experience; it cannot repair a stock gap, a late delivery or a slow reply.
- Never holding anyone out. Without a comparison group, every purchase by a member looks like the programme's success.
- Rewarding reorders a subscription would have secured. A consumable on a steady cycle suits a standing order, not points after the fact.
- Celebrating breakage. Unredeemed rewards save money and signal that the programme is motivating nobody.
- Judging the scheme on a blended rate. The long-standing customers who dominate it hide what members' cohorts actually do.
- Changing terms or balances quietly. Unclear expiry and quiet devaluations erode trust and create regulatory risk.
