Why a plan needs a learning agenda
The marketing plan tutorial produces a document that says what the business will do and how it will know. Behind each line sits an assumption the plan treats as true. The target segment is assumed to be reachable and interested; the forecast assumes a conversion rate; the budget split assumes channels will deliver at a certain cost; the calendar assumes the team and the operation can cope. Some of those assumptions rest on years of evidence. Others are guesses dressed as numbers.
A test backlog does not fix this on its own. The experimentation cadence guide and the playbook run a pipeline of ideas well, but ideas tend to come from whatever is easiest to test, such as headlines, buttons and subject lines, not from the assumptions the plan most depends on. The learning agenda works one level up. It starts from the plan, finds the assumptions that would change the plan if they were wrong, and makes sure each has an answer before the decision that rests on it is taken.
Three things distinguish a learning agenda from a list of tests:
- It starts from decisions, not ideas. Every item exists because a decision in the plan depends on it.
- It uses any method that answers the question. Existing data, research, an experiment, a holdout or a pilot, whichever is cheapest and adequate.
- It has dates. Each answer is scheduled to arrive before its decision is due, which is what makes it part of the plan rather than an afterthought.
The assumptions tend to cluster in the same parts of every plan, and each part has methods that usually suit it.
| Part of the plan | Typical assumption | Often answered by |
|---|---|---|
| Segment | This group is large enough, reachable and has the need we describe | Existing customer data, research, a small paid test aimed at the segment |
| Message | This benefit is what persuades them | Customer interviews, a split test of ads or pages |
| Channel | This channel delivers the audience at the cost the budget assumes | Past results, a bounded test, a holdout for channels already running |
| Price and offer | Customers buy at this price; this offer is attractive enough | Sales and booking history, a test of offers, a pilot in one market |
| Conversion | The site, sales team or booking flow converts at the forecast rate | Analytics and the sales ledger, a page or form test |
| Capacity | The business can serve the demand the plan creates | Operations data, a pilot at one location or in one period |
Some uncertainty cannot be tested at all. Whether a competitor will cut prices, or whether demand across the market will fall, is not an assumption the business can check in advance; the scenario planning framework prepares for futures like those. The learning agenda handles the assumptions the business can check. The academy capstone on a strategy leadership plan for a sauna maker practises writing a learning agenda that tests the plan's assumptions beside its market portfolio, crisis readiness and agency integration.
Before you start
- The plan itself. A written plan, or at least its objectives, segments, messages, channel mix, budget, forecast and calendar. The annual planning guide sets out what the year's plan contains and when its decisions fall.
- The decision calendar. When each significant commitment has to be made: media bookings, launch dates, stock orders, hiring, contract renewals, the quarterly reviews where budget moves.
- What the business already knows. Past campaign results, sales and booking data, customer research, previous test results and the learning logA single, searchable record of every experiment, losers and inconclusive results included, with the hypothesis, design, result, decision and what it taught. if one exists.
- The people who made the plan. The assumptions are clearest to the people who wrote the numbers. Include someone from finance or operations, who will see assumptions marketing takes for granted.
- A place for the agenda. One table, in the same place as the plan, with an owner who keeps it current. It sits beside the brief, the calendar and the budget described in the planning section, and those documents should point to the agenda items their numbers depend on.
Allow a working session for the first draft, then time over the following week for the owner to check what existing data can answer before any new work is commissioned.
The steps
List the assumptions behind each part of the plan
Go through the plan section by section and ask, for every target, forecast and choice, what has to be true for it to work. Write each assumption as a plain statement that could turn out to be false. Cover every part of the plan: the segment (this group exists in the numbers we think, can be reached, and has the need we describe), the message (this benefit is what moves them), the channel (this channel can deliver this audience at roughly the cost the budget assumes), the price and offer (customers will buy at this price, this offer is attractive enough), conversion (the site, the sales team or the booking flow converts at the rate in the forecast) and capacity (the business can serve the demand the plan creates). Expect a long list. Do not judge the assumptions yet; the point is to make them visible.
Rank them by consequence and confidence
For each assumption, ask two questions. If this is wrong, how much of the plan fails? A wrong assumption about the main segment can undo the whole year; a wrong guess about the best day to send email costs very little. How confident are we, and on what evidence? Several years of the business's own data is strong; a supplier's claim or a competitor's apparent success is weak. Score both on a simple scale and plot them. The assumptions that are high in consequence and low in confidence are the agenda's core. The impact-effort matrix uses the same two-axis thinking for work; here the axes are the cost of being wrong and the strength of the evidence. Assumptions with high confidence and low consequence can be accepted and left alone.
Mark which decisions are hard to reverse
Next to each important assumption, note the decision it informs and how hard that decision is to undo. The one-way and two-way doors framework sorts decisions this way: an email subject line or a bid change is a two-way door that can be tried and reversed; a year-long contract, a new location, a rebrand or a stock commitment is a one-way door. One-way doors deserve evidence before they are walked through, so the assumptions behind them move up the agenda even when their confidence looks reasonable. The framework also suggests making one-way doors smaller, by piloting in one place, holding some customers back or time-limiting the commitment, which is often where the method in the next step comes from.
Choose the cheapest method that would answer each
For each assumption on the agenda, choose the least expensive method that would give an answer good enough for the decision. Look it up: orders, bookings, search data and past campaigns often already hold the answer. Ask: interviews or a survey, designed from the decision backwards as the research design guide sets out. Test: a controlled experiment, such as an A/B TestControlled comparison of two variants. on a page or a split of an email list, drawing on the designs in the experiment library. Hold out: withhold a channel or campaign from a comparable group to measure what it causes, as the incrementality testing guide describes, using a Holdout GroupA randomly chosen part of an audience kept from exposure to measure what exposure caused.. Pilot: try the change in one region, location, product or segment before committing everywhere. Prefer the cheaper method whenever it is adequate; reserve experiments and holdouts for questions that only they can answer.
Write each as a hypothesis tied to a decision
Rewrite each assumption that needs active work as a HypothesisA testable statement of the change to make, the effect expected on a named metric and the evidence-based reason for expecting it.: what change or condition, what effect on which measure, for whom, and why. Then add the line most test documents leave out: what decision this answer will change, and how. For example, if the pilot shows the result, the plan does one thing; if it does not, it does another. The experiment hypothesis template gives the fields for a test, including the decision rule written before the test starts. Use the same discipline for research and pilots. An item whose answer would not change any decision does not belong on the agenda, however interesting it is.
Schedule each answer before its decision date
Work backwards from each decision date. A check that reports after the commitment is made is history, not learning. The backward planning framework gives each check a latest start counted back from its decision date. Allow for the time to set up the method, run it, wait for results that arrive late (orders after a click, bookings after an enquiry) and discuss the answer. Where the answer cannot arrive in time, change something: choose a faster method, move the decision, make the decision smaller, or accept the risk explicitly and record that the plan is proceeding on an untested assumption. Put the schedule beside the annual plan's calendar so both are read together.
Fix sample and duration before starting
For every experiment, holdout and pilot, decide in advance how large the sample will be, how long the check will run and how the result will be read, then run to that plan. Stopping early when a result looks good is the commonest way to learn something false. The site's testing guides cite Evan Miller's essay on how not to run an A/B test for this warning: a test checked repeatedly and stopped the first time it looks significant declares false winners far more often than its nominal error rate suggests. The sample size entry and the sample size and significance guide explain how to size a test from the traffic the business has. If the Sample SizeThe number of users or sessions an experiment needs to detect a given effect reliably. needed is out of reach, use a larger, simpler change, a longer period or a different method, rather than running an underpowered test and reading noise.
Review the agenda monthly
Once a month, the owner walks through the agenda with the people who own the plan. For each item: is it on schedule, has an answer arrived, has the decision date moved? Add assumptions the plan has acquired since the last review, such as a new channel, a competitor's move or a price change, and retire items that no longer matter. The experimentation loop describes the cycle from observation to tested change to new observation; the monthly review is where the loop meets the plan. Keep the review short and the table current; an agenda that is updated only at year end has stopped working.
Feed the answers back into the plan
An answer that does not change the plan has been wasted. When an item reports, record the result, the decision it led to and the change made to the plan: a segment dropped, a budget moved, a forecast revised, a launch delayed. Then update the confidence on related assumptions, because one answer often strengthens or weakens several. Record results that did not support the hypothesis as carefully as those that did; the playbook's learning log is the place for both. At the next annual planning cycle, start from the agenda's record: what the business now knows, what it still assumes and what it learned too late.
A yoga studio group's agenda for the year (illustrative)
A group of yoga studios builds its first learning agenda
Mistlecove Yoga, an invented group of 5 yoga studios, plans its year around growth from office workers: a new lunchtime timetable in all 5 studios from April, a £3,000 a month search and social budget aimed at people working nearby, and an introductory pass priced at £39 for 3 weeks. The marketing lead and the operations manager list 14 assumptions in one session and rank them.
| Assumption | Consequence if wrong | Confidence | Door | Method | Decision it informs | Answer due |
|---|---|---|---|---|---|---|
| Office workers will attend lunchtime classes | High: the timetable and the year's growth rest on it | Low: no lunchtime classes have run | One-way: timetable and teacher contracts for the year | Pilot in 2 studios for 6 weeks | Roll the timetable out to all 5 studios or not | End of March |
| Intro pass buyers convert to memberships | High | Medium: booking data exists | Two-way | Look up: last year's bookings | Whether to keep the pass at £39 | End of January |
| Search can reach office workers nearby at an acceptable cost | Medium | Low | Two-way | Test: 4 weeks of search in the pilot areas | The search share of the budget | End of March |
| Members renew after a price rise | Medium | Medium | Two-way | Look up: renewals after last year's rise | The price plan for autumn | End of February |
The cheaper questions are answered from data the studios already hold. Booking records show that 31% of intro pass buyers took a membership last year, against 12% of drop-in visitors, so the pass stays. Renewals barely moved after the last price rise.
The lunchtime pilot runs in the two studios nearest office areas for 6 weeks, with its success rule written in advance: an average of at least 8 people per lunchtime class by week 6, the level at which a class covers the teacher's fee. The team agrees not to judge it earlier. Classes average 5 in week 2, which tempts the lead to stop, and 9 by week 6 as word spreads through offices. Search in the pilot areas brings intro pass sales at about £24 each, inside the £30 the plan allowed. In April the timetable goes to 4 studios, not 5: the fifth has no offices within walking distance, which the pilot made obvious, and its lunchtime slot becomes a second evening class instead.
Common mistakes
- Testing what is easy instead of what matters. Headlines and buttons are easy to test; the plan usually depends on bigger assumptions about segments, channels and price.
- Running a test when the data already holds the answer. Look it up before commissioning anything.
- Answers that arrive after the decision. Schedule backwards from decision dates, or change the decision.
- Hypotheses with no decision attached. If no answer would change what the business does, drop the item.
- Stopping a test when the result looks good. Fix the sample and duration first and run to them.
- Treating one-way doors like two-way doors. Hard-to-reverse commitments need evidence or a smaller first step.
- Ignoring results that disappoint. A hypothesis not supported is still an answer; record it and change the plan.
- An agenda reviewed once a year. Monthly reviews keep it tied to the plan as both change.
