Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Growth Marketing › Advanced

Reading Incremental Sign-ups for a Budgeting App

You read a finished budget-increase test for an invented budgeting app that raised install spend by two different amounts in matched regions. The exercise practises turning regional results into incremental sign-ups and paying users, comparing them with what the platforms credited, finding the step at which the marginal paying user costs more than one is worth, and writing the budget recommendation and the next test.
ExerciseAdvanced levelModule 2: Incremental growth and marginal spend5 minUpdated

Why this matters

A growth team that scales on credited sign-ups scales into its own organic demand. The question a budget decision asks is narrower: what did the last slice of spend cause, and is a user bought at that price worth it? The budget increase incrementality test is the design that answers it, the incrementality testing guide explains how lift is read, and the marginal returns and forecasting guide explains why the last slice matters more than the average. At the intermediate level you built acquisition cost and payback from cohorts; here those figures become the threshold the test is judged against.

The app and its budget test

Coinwick Money is an invented personal budgeting app with a free tier and a paid subscription. It acquires users through app-install campaigns. To learn whether more spend would buy more paying users, it matched eighteen regions into groups of three on pre-period sign-ups and, within each group, randomly left one region alone, raised spend by about a third in another and by about three-fifths in the third.

Illustrative example — not a real result

Coinwick Money: the budget test, six weeks after an eight-week pre-period

CellPre-period sign-upsSpend in test weeks (planned)Spend in test weeks (actual)Sign-ups in test weeksPaying users within 30 days
Control5,000£15,000£15,0005,4001,080
Cell A, about +33%4,500£13,500£18,0005,2401,022
Cell B, about +60%4,500£13,500£21,6005,4001,042
  • Sign-ups the platforms credited in each test cell: about 1,700 at planned spend; during the test, 2,500 in A and 2,950 in B.
  • Contribution per paying user within the twelve-month payback window: £110 on average.
  • Paying users in the test cells kept their subscription into month 2 at 71%, against 78% for paying users generally.
  • Current national install budget: £90,000 a quarter.

From attributed to incremental to marginal

Three readings, in order. Incremental: the sign-ups each test cell gained over what the control cell predicts from the pre-period relationship. Paying: the same for paying users, because a free sign-up is not a customer and counting it makes acquisition cost (CAC) and value look low in the same stroke, as the LTV and CAC guide warns. Marginal: the extra spend of each step divided by the extra paying users that step bought, compared with what a marginal paying user is worth. Users bought at the margin are often less valuable than the average, so check before assuming they are not.

Task

  1. Predict each test cell's sign-ups and paying users without the increase, and calculate the incremental figures.
  2. Compare incremental sign-ups with the extra sign-ups the platforms credited.
  3. Calculate the marginal cost per incremental sign-up and per incremental paying user for each step.
  4. Estimate what a marginal paying user is worth and compare.
  5. Recommend next quarter's budget and state the caveats.
  6. Write the next test.

Worked answer

Illustrative example — not a real result

Coinwick's test, read

Pre-period ratio: 4,500 ÷ 5,000 = 0.9 for each test cell. Predicted sign-ups: 0.9 × 5,400 = 4,860. Predicted paying users: 0.9 × 1,080 = 972.

Row labelCell ACell BStep from A to B
Extra spend£4,500£8,100£3,600
Incremental sign-ups380540160
Extra sign-ups the platforms credited8001,250450
Incremental paying users507020
Paying share of incremental sign-ups≈ 13%≈ 13%12.5%
Marginal cost per incremental sign-up≈ £11.84£15.00£22.50
Marginal cost per incremental paying user£90≈ £116£180

Platform-credited cost per sign-up during the test: A £18,000 ÷ 2,500 = £7.20; B £21,600 ÷ 2,950 ≈ £7.32. Paying share of all control sign-ups: 1,080 ÷ 5,400 = 20%. Value of a marginal paying user, adjusted for weaker early retention: £110 × 71 ÷ 78 ≈ £100.

Attributed against incremental. The platforms credited roughly twice the sign-ups the extra spend caused, and their cost per sign-up barely moved between the two steps. The dashboard says the second step was as good as the first; the test says it was far worse.

Quality at the margin. Incremental sign-ups paid at about two-thirds the usual rate and lapsed faster in their first month. Marginal users are cheaper to count and dearer to keep, which is why the comparison runs on paying users and on a value adjusted for their retention, not on the average lifetime value (LTV).

Recommendation. The first step bought paying users below what they are worth; the second step's users cost nearly twice their value. Raise the national budget by about a third next quarter, roll it out region by region rather than at once, and keep a few regions at the old level as a standing control. Report the expected gain as a range, because the result rests on six weeks and a modest number of incremental payers, and the true Incrementality of the step could sit well either side of the midpoint.

Caveats. The thirty-day paying window misses slower converters, which would lower the marginal cost; the weaker retention would raise it. Re-read both cells at sixty days before the rollout passes the halfway point.

Next test. Hold the new level and test a further step of about half the size of the failed one, so the crossing point between the two levels is found. Run a second cell that changes creative towards people likely to pay, because raising the paying share of marginal sign-ups would move the crossing point further than any budget change.

Common mistakes

  • Judging the test on credited sign-ups. The dashboard's cost per sign-up hides the difference between steps.
  • Comparing the marginal cost with the average value. Marginal users retain worse; adjust before comparing.
  • Counting free sign-ups as customers. The threshold is set on paying users.
  • Scaling to the largest cell because its total gain was biggest. The question is what the last step bought.
  • Removing every control region at rollout. Without one, the next drift goes unseen.

Pages this lesson applies

Terms used on this page

Also related

Related