Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Digital Marketing › Planning & Budgeting

Turning Plan Assumptions into a Learning Agenda

How to list the assumptions a marketing plan rests on, rank them by risk, choose a way to check each and schedule the answers before the plan depends on them.
Tutorialintermediate11 min readUpdated

Why a plan needs a learning agenda

The marketing plan tutorial produces a document that says what the business will do and how it will know. Behind each line sits an assumption the plan treats as true. The target segment is assumed to be reachable and interested; the forecast assumes a conversion rate; the budget split assumes channels will deliver at a certain cost; the calendar assumes the team and the operation can cope. Some of those assumptions rest on years of evidence. Others are guesses dressed as numbers.

A test backlog does not fix this on its own. The experimentation cadence guide and the playbook run a pipeline of ideas well, but ideas tend to come from whatever is easiest to test, such as headlines, buttons and subject lines, not from the assumptions the plan most depends on. The learning agenda works one level up. It starts from the plan, finds the assumptions that would change the plan if they were wrong, and makes sure each has an answer before the decision that rests on it is taken.

Three things distinguish a learning agenda from a list of tests:

  • It starts from decisions, not ideas. Every item exists because a decision in the plan depends on it.
  • It uses any method that answers the question. Existing data, research, an experiment, a holdout or a pilot, whichever is cheapest and adequate.
  • It has dates. Each answer is scheduled to arrive before its decision is due, which is what makes it part of the plan rather than an afterthought.

The assumptions tend to cluster in the same parts of every plan, and each part has methods that usually suit it.

Part of the planTypical assumptionOften answered by
SegmentThis group is large enough, reachable and has the need we describeExisting customer data, research, a small paid test aimed at the segment
MessageThis benefit is what persuades themCustomer interviews, a split test of ads or pages
ChannelThis channel delivers the audience at the cost the budget assumesPast results, a bounded test, a holdout for channels already running
Price and offerCustomers buy at this price; this offer is attractive enoughSales and booking history, a test of offers, a pilot in one market
ConversionThe site, sales team or booking flow converts at the forecast rateAnalytics and the sales ledger, a page or form test
CapacityThe business can serve the demand the plan createsOperations data, a pilot at one location or in one period

Some uncertainty cannot be tested at all. Whether a competitor will cut prices, or whether demand across the market will fall, is not an assumption the business can check in advance; the scenario planning framework prepares for futures like those. The learning agenda handles the assumptions the business can check. The academy capstone on a strategy leadership plan for a sauna maker practises writing a learning agenda that tests the plan's assumptions beside its market portfolio, crisis readiness and agency integration.

Before you start

  • The plan itself. A written plan, or at least its objectives, segments, messages, channel mix, budget, forecast and calendar. The annual planning guide sets out what the year's plan contains and when its decisions fall.
  • The decision calendar. When each significant commitment has to be made: media bookings, launch dates, stock orders, hiring, contract renewals, the quarterly reviews where budget moves.
  • What the business already knows. Past campaign results, sales and booking data, customer research, previous test results and the learning log if one exists.
  • The people who made the plan. The assumptions are clearest to the people who wrote the numbers. Include someone from finance or operations, who will see assumptions marketing takes for granted.
  • A place for the agenda. One table, in the same place as the plan, with an owner who keeps it current. It sits beside the brief, the calendar and the budget described in the planning section, and those documents should point to the agenda items their numbers depend on.

Allow a working session for the first draft, then time over the following week for the owner to check what existing data can answer before any new work is commissioned.

The steps

  1. List the assumptions behind each part of the plan

    Go through the plan section by section and ask, for every target, forecast and choice, what has to be true for it to work. Write each assumption as a plain statement that could turn out to be false. Cover every part of the plan: the segment (this group exists in the numbers we think, can be reached, and has the need we describe), the message (this benefit is what moves them), the channel (this channel can deliver this audience at roughly the cost the budget assumes), the price and offer (customers will buy at this price, this offer is attractive enough), conversion (the site, the sales team or the booking flow converts at the rate in the forecast) and capacity (the business can serve the demand the plan creates). Expect a long list. Do not judge the assumptions yet; the point is to make them visible.

  2. Rank them by consequence and confidence

    For each assumption, ask two questions. If this is wrong, how much of the plan fails? A wrong assumption about the main segment can undo the whole year; a wrong guess about the best day to send email costs very little. How confident are we, and on what evidence? Several years of the business's own data is strong; a supplier's claim or a competitor's apparent success is weak. Score both on a simple scale and plot them. The assumptions that are high in consequence and low in confidence are the agenda's core. The impact-effort matrix uses the same two-axis thinking for work; here the axes are the cost of being wrong and the strength of the evidence. Assumptions with high confidence and low consequence can be accepted and left alone.

  3. Mark which decisions are hard to reverse

    Next to each important assumption, note the decision it informs and how hard that decision is to undo. The one-way and two-way doors framework sorts decisions this way: an email subject line or a bid change is a two-way door that can be tried and reversed; a year-long contract, a new location, a rebrand or a stock commitment is a one-way door. One-way doors deserve evidence before they are walked through, so the assumptions behind them move up the agenda even when their confidence looks reasonable. The framework also suggests making one-way doors smaller, by piloting in one place, holding some customers back or time-limiting the commitment, which is often where the method in the next step comes from.

  4. Choose the cheapest method that would answer each

    For each assumption on the agenda, choose the least expensive method that would give an answer good enough for the decision. Look it up: orders, bookings, search data and past campaigns often already hold the answer. Ask: interviews or a survey, designed from the decision backwards as the research design guide sets out. Test: a controlled experiment, such as an A/B Test on a page or a split of an email list, drawing on the designs in the experiment library. Hold out: withhold a channel or campaign from a comparable group to measure what it causes, as the incrementality testing guide describes, using a Holdout Group. Pilot: try the change in one region, location, product or segment before committing everywhere. Prefer the cheaper method whenever it is adequate; reserve experiments and holdouts for questions that only they can answer.

  5. Write each as a hypothesis tied to a decision

    Rewrite each assumption that needs active work as a Hypothesis: what change or condition, what effect on which measure, for whom, and why. Then add the line most test documents leave out: what decision this answer will change, and how. For example, if the pilot shows the result, the plan does one thing; if it does not, it does another. The experiment hypothesis template gives the fields for a test, including the decision rule written before the test starts. Use the same discipline for research and pilots. An item whose answer would not change any decision does not belong on the agenda, however interesting it is.

  6. Schedule each answer before its decision date

    Work backwards from each decision date. A check that reports after the commitment is made is history, not learning. The backward planning framework gives each check a latest start counted back from its decision date. Allow for the time to set up the method, run it, wait for results that arrive late (orders after a click, bookings after an enquiry) and discuss the answer. Where the answer cannot arrive in time, change something: choose a faster method, move the decision, make the decision smaller, or accept the risk explicitly and record that the plan is proceeding on an untested assumption. Put the schedule beside the annual plan's calendar so both are read together.

  7. Fix sample and duration before starting

    For every experiment, holdout and pilot, decide in advance how large the sample will be, how long the check will run and how the result will be read, then run to that plan. Stopping early when a result looks good is the commonest way to learn something false. The site's testing guides cite Evan Miller's essay on how not to run an A/B test for this warning: a test checked repeatedly and stopped the first time it looks significant declares false winners far more often than its nominal error rate suggests. The sample size entry and the sample size and significance guide explain how to size a test from the traffic the business has. If the Sample Size needed is out of reach, use a larger, simpler change, a longer period or a different method, rather than running an underpowered test and reading noise.

  8. Review the agenda monthly

    Once a month, the owner walks through the agenda with the people who own the plan. For each item: is it on schedule, has an answer arrived, has the decision date moved? Add assumptions the plan has acquired since the last review, such as a new channel, a competitor's move or a price change, and retire items that no longer matter. The experimentation loop describes the cycle from observation to tested change to new observation; the monthly review is where the loop meets the plan. Keep the review short and the table current; an agenda that is updated only at year end has stopped working.

  9. Feed the answers back into the plan

    An answer that does not change the plan has been wasted. When an item reports, record the result, the decision it led to and the change made to the plan: a segment dropped, a budget moved, a forecast revised, a launch delayed. Then update the confidence on related assumptions, because one answer often strengthens or weakens several. Record results that did not support the hypothesis as carefully as those that did; the playbook's learning log is the place for both. At the next annual planning cycle, start from the agenda's record: what the business now knows, what it still assumes and what it learned too late.

A yoga studio group's agenda for the year (illustrative)

Illustrative example — not a real result

A group of yoga studios builds its first learning agenda

Mistlecove Yoga, an invented group of 5 yoga studios, plans its year around growth from office workers: a new lunchtime timetable in all 5 studios from April, a £3,000 a month search and social budget aimed at people working nearby, and an introductory pass priced at £39 for 3 weeks. The marketing lead and the operations manager list 14 assumptions in one session and rank them.

AssumptionConsequence if wrongConfidenceDoorMethodDecision it informsAnswer due
Office workers will attend lunchtime classesHigh: the timetable and the year's growth rest on itLow: no lunchtime classes have runOne-way: timetable and teacher contracts for the yearPilot in 2 studios for 6 weeksRoll the timetable out to all 5 studios or notEnd of March
Intro pass buyers convert to membershipsHighMedium: booking data existsTwo-wayLook up: last year's bookingsWhether to keep the pass at £39End of January
Search can reach office workers nearby at an acceptable costMediumLowTwo-wayTest: 4 weeks of search in the pilot areasThe search share of the budgetEnd of March
Members renew after a price riseMediumMediumTwo-wayLook up: renewals after last year's riseThe price plan for autumnEnd of February

The cheaper questions are answered from data the studios already hold. Booking records show that 31% of intro pass buyers took a membership last year, against 12% of drop-in visitors, so the pass stays. Renewals barely moved after the last price rise.

The lunchtime pilot runs in the two studios nearest office areas for 6 weeks, with its success rule written in advance: an average of at least 8 people per lunchtime class by week 6, the level at which a class covers the teacher's fee. The team agrees not to judge it earlier. Classes average 5 in week 2, which tempts the lead to stop, and 9 by week 6 as word spreads through offices. Search in the pilot areas brings intro pass sales at about £24 each, inside the £30 the plan allowed. In April the timetable goes to 4 studios, not 5: the fifth has no offices within walking distance, which the pilot made obvious, and its lunchtime slot becomes a second evening class instead.

Common mistakes

  • Testing what is easy instead of what matters. Headlines and buttons are easy to test; the plan usually depends on bigger assumptions about segments, channels and price.
  • Running a test when the data already holds the answer. Look it up before commissioning anything.
  • Answers that arrive after the decision. Schedule backwards from decision dates, or change the decision.
  • Hypotheses with no decision attached. If no answer would change what the business does, drop the item.
  • Stopping a test when the result looks good. Fix the sample and duration first and run to them.
  • Treating one-way doors like two-way doors. Hard-to-reverse commitments need evidence or a smaller first step.
  • Ignoring results that disappoint. A hypothesis not supported is still an answer; record it and change the plan.
  • An agenda reviewed once a year. Monthly reviews keep it tied to the plan as both change.

Frequently asked questions

What is the difference between a learning agenda and a test backlog?

A test backlog is a list of ideas waiting to be run, usually ranked by expected impact and effort. A learning agenda sits above it: it starts from the plan, lists the assumptions the plan cannot afford to get wrong, and decides which of them need an answer and by when. Some of those answers come from tests in the backlog; many come from data the business already holds, from research or from a pilot. The agenda tells the backlog which tests matter to the plan.

How many assumptions should a learning agenda include?

As many as the plan genuinely rests on, but only a few will be worth active work at any time. Rank them, answer the cheap ones from existing data straight away, and spend real effort only on the assumptions that are both consequential and uncertain, starting with those whose decision date comes first. An agenda with more open checks than the team can run properly is a list of good intentions.

Can a small business run a learning agenda without A/B testing?

Yes. Many assumptions are answered from orders, bookings, enquiries and search data the business already has, or from talking to customers. Where a test is needed, a small business often lacks the traffic for a split test that can detect a realistic difference, and a pilot in one location, one product or one period, compared honestly with what came before, is the practical alternative. The discipline is the same: write down what would change the decision before looking at the results.

Terms used on this page

Frameworks

Where this page is used

Part of these learning paths

Sources

  1. How Not To Run an A/B Test — Evan Miller

Related