Hypothesis
If a random half of inbound demo and contact requests are called back immediately instead of through the standard queue, more of them will book and hold a meeting, because the buyer is still engaged and has not yet spoken to a competitor. The decision behind it: staff and route for immediate follow-up of inbound leads, or keep the standard queue.
The case for speed is intuitive, and the inbound lead handling playbook sets response standards on that basis. An immediate-response rota is not free, though. It takes rep time away from other work, needs cover across the hours leads arrive, and may simply move meetings forward without adding any. The test asks whether speed adds meetings held for this business's buyers, and whether enough are added to pay for the rota. Write down the rota's hours and its target response, the standard queue's current target, and the cost of the rota, in the experiment hypothesis template before launch.
Metrics and guardrails
The primary metric is meetings held per inbound lead: of all leads assigned to an arm, the share that has held a first meeting with sales by the end of the follow-up window. The denominator includes leads that could not be reached, so a rota that calls fast but misses people gets no credit for the calls. Use the meeting stage as it is defined in the CRM, following the lead stages and pipeline guide, and agree with sales which records count as held.
Three guardrails protect the decision. The sales-qualified lead rate among held meetings shows whether speed brings extra meetings with people who are unlikely to buy; if it falls, the gain is weaker than it looks. Rep hours per lead show the cost of the rota, which the decision rule needs. The third guardrail is the response time in the standard arm, which must not slip during the test: if the rota draws people away from the queue, the standard arm gets slower than usual, and the test then compares fast with unusually slow rather than fast with normal. If the business tracks marketing-qualified leadsA lead that marketing judges ready for sales follow-up against agreed criteria. separately from demo requests, decide in advance which lead types enter the test.
Design
Randomise at the level of the inbound lead, at form submission, using a routing rule in the CRM or the marketing automation platform. Leads in the immediate arm go to the rota with a target of calling within the agreed few minutes; leads in the standard arm go to the queue exactly as today. Everything else, the form, the confirmation page, the automated email and the sales process after the first contact, stays identical.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Inbound lead, assigned at form submission | Speed is decided at the moment the lead arrives |
| Arms | Immediate-response rota versus standard queue | One change: how quickly the first call happens |
| Eligible leads | Demo and contact requests arriving in rota hours | The test answers what the business can actually staff |
| Lead visibility | Reps work only their own arm's leads | Prevents cherry-picking across arms |
| Primary metric | Meetings held per lead within a fixed window | Counts outcomes, not calls or bookings |
| Standard arm | Monitored against its usual response time | The control must stay normal |
Separating the work is the hardest part. If the same reps handle both arms, they will naturally reach first for the most promising leads, whichever arm they are in. Give the rota its own people for its hours, and hide immediate-arm leads from the standard queue and the reverse. The sales and marketing alignment guide covers how to agree service levels and shared definitions with sales; a short written agreement on the test's rules belongs there too.
Sample size reasoning
Inbound leads are few compared with web visitors, so the sample is the main constraint. The inputs are the current meetings held per lead, the smallest gain that would pay for the staffing, and the confidence required; the sample size and significance guide shows how they set the leads each arm needs. The smallest gain worth detecting is a cost question: convert the rota's hours into a cost, the value of a held meeting into a figure using the lead-to-customer rate and deal value, and find the extra meetings needed to break even. Expect a test of weeks or months. A business with only a handful of leads a week should consider whether a test can ever settle the question, or whether the playbook's standards are a sounder default.
Duration
Enrol across full weekly cycles until the planned sample is reached, because leads arriving on a Monday morning and a Friday afternoon behave differently. Then allow a follow-up window long enough for meetings to be held, based on how long meetings usually take to schedule; for complex sales with a long sales cycleAverage time from opportunity creation to a closed deal., the first meeting is still the right read point, with pipeline checked later as a secondary view. Avoid running across a major campaign launch or an event that floods the queue.
Reading the results and the decision rule
Read once, after the follow-up window closes for the last enrolled lead. Compare meetings held per lead with its confidence interval, then the guardrails. The rule, written before launch:
| Result | Decision |
|---|---|
| Meetings held per lead rise enough to cover the rota's cost, and qualification holds | Fund immediate response |
| Meetings rise but qualification falls | Fund a narrower rota for the highest-intent lead types only |
| No reliable difference, or a gain too small to pay for the rota | Tighten the standard queue's target instead |
| Standard arm's response time slipped during the test | Treat the result as unreliable and rerun with separate capacity |
Once the test has finished, a significance reader for two conversion rates confirms the primary read. Enter leads in each arm as visitors and meetings held as conversions. It reads a completed test; it does not size one.
Lead response significance checker
A warehouse software firm tests a callback rota
Palletquay Systems, an invented warehouse management software company, receives about 75 inbound demo and contact requests a week. For sixteen weeks it routes 600 of them at random to a two-person rota that calls within five minutes in office hours, and 600 to the standard queue, which aims for the same working day. After a three-week follow-up window, meetings held per lead are 31% for the rota and 24% for the queue, with a 95% interval for the difference of roughly 2 to 12 points. The sales-qualified rate among held meetings is level, and the queue kept its usual response time. The extra meetings outweigh the rota's cost, so Palletquay funds it.
Pitfalls
Reps cherry-picking leads across arms is the first pitfall, and it can create a difference where none exists or hide one that does. Separate people and queues by arm. The standard arm slowing because the rota takes capacity is the second: if the queue's response time drifts during the test, the comparison is no longer against normal practice. Monitor it weekly and treat a breach as a design failure. Out-of-hours leads mixed in are the third. A rota cannot call immediately at night, so leads arriving outside its hours dilute the immediate arm with leads it treated no differently; exclude them from randomisation from the start.
Where follow-up still fails after the test, for reasons of ownership or capacity rather than speed, the guide to sales not following up on marketing leads helps diagnose the cause.
Common mistakes
Speed-to-lead tests go wrong when the arms share people or the wrong outcome is counted. The lead form length test changes the form before submission rather than the response after it, and the experiment library lists every design.
- Comparing fast and slow responses from CRM history. Fast responses go to better-looking leads; randomise.
- Letting one team work both arms. Reps will cherry-pick across them.
- Counting calls or bookings. Meetings held are the outcome sales time buys.
- Including leads the rota cannot reach in time. Restrict the test to rota hours.
- Ignoring the rota's cost. A gain too small to pay for the staffing is not a win.
- Letting the standard queue slow down. The control must stay at its normal pace.
