Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Demo Scheduling Test

An experiment design for demo requests: instant calendar booking after the form against a sales callback, read on demos held per request.
Experimentintermediate6 min readUpdated

Hypothesis

If demo requesters can book a slot on a calendar immediately after the form instead of waiting for a sales callback, more requests will turn into held demos, because the commitment is made while intent is highest. The decision behind it: let demo requesters book a slot immediately after the form, or have sales call them back to arrange it.

The callback has strengths the calendar lacks. A rep can qualify the requester before committing time, steer an unsuitable prospect elsewhere and build some rapport before the demo. The calendar removes a round of phone or email tag and lets the requester commit at the moment they asked. Which effect is larger depends on the buyers, the sales team's speed and how many unsuitable companies request demos. The inbound lead handling playbook describes how demo requests should be handled in general; this test settles the specific choice of mechanism. Record the arms, the contact rules for each and the guardrail limits in the experiment hypothesis template before launch.

Metrics and guardrails

The primary metric is demos held per demo request: of all requests assigned to an arm, the share that results in a demo that actually took place within the follow-up window. Bookings per request and the confirmation step's CVR are reported as context, but they are what the calendar arm would naturally win, so they do not decide the test. The denominator is every request, including those the callback arm never reached.

Three guardrails catch the ways instant booking can disappoint. The no-show rate among booked demos shows how many easy bookings were never meant seriously; a rise is expected, and the question is whether held demos still gain. The sales-qualified rate of held demos shows whether the calendar fills sales time with weaker prospects. Bookings from companies outside the target profile, judged against the profile sales already uses, catch the unsuitable demos a callback would have screened out. Agree with sales, through the shared definitions described in the sales and marketing alignment guide, which records count as held and qualified before launch.

Design

Randomise at the level of the demo requester, at form submission. In the calendar arm, the confirmation step shows a booking calendar with the next available slots and a clear CTA to confirm one; an automated confirmation and reminder follow. In the callback arm, the confirmation step says sales will be in touch, and the standard callback process runs. The form, its fields and its wording stay identical. The forms guide covers form design; usability research such as the Nielsen Norman Group's recommendations for web forms is a useful check that the form itself is not the bottleneck before this test begins. The calendar arm is a small booking flow, and the guide to booking and appointment flows covers how such a flow shows availability, confirms the slot and makes changes easy.

Design elementChoiceReason
Unit of randomisationDemo requester, assigned at form submissionThe step after the form is what differs
ArmsBooking calendar on the confirmation step versus standard callbackOne change: how the demo gets arranged
FormIdentical in both armsIsolates the post-form step
Contact rulesCalendar arm gets confirmation and reminders, a call only if missedPrevents the calendar arm getting both treatments
AvailabilitySame reps and similar slot availability for both arms each weekCalendar gaps would penalise one arm
Primary metricDemos held per request within a fixed windowCounts conversations, not bookings

Sample size reasoning

Demo requests are scarce, so the sample is the main constraint. The inputs are the current share of requests that end in a held demo, the smallest gain that would justify changing the process and the confidence required. The sample size and significance guide shows how they set the requests each arm needs, and the conversion rate calculator set up for lead generation helps derive the baseline from your own form and CRM figures. A small team receiving few requests may need months; if the planned sample is out of reach, consider whether a larger difference, such as instant booking against a callback that is slower than usual, is the question worth testing.

Duration

Enrol across full weekly cycles until the planned sample is reached, then allow a window long enough for booked demos to take place and for the callback arm to arrange and hold its demos. The callback arm's demos usually happen later, so reading before the window closes favours the calendar. Avoid enrolling across holidays or events that change reps' availability for part of the test.

Reading the results and the decision rule

Read once, after the follow-up window closes for the last enrolled request. Compare demos held per request with its confidence interval, then each guardrail. The rule, written before launch:

ResultDecision
Demos held per request rise and qualification does not fallAdopt instant booking
Demos held rise but out-of-profile bookings growAdopt instant booking with a qualifying step before the calendar
Demos held rise but the qualified rate of held demos fallsKeep the callback, or offer the calendar only to requests matching the target profile
No reliable differenceKeep whichever process sales prefers to run

Once the test has finished, a significance reader for two conversion rates confirms the primary read. Enter requests in each arm as visitors and demos held as conversions. It reads a completed test; it does not size one.

Demo scheduling significance checker

Held demos lead to customers at the lead-to-customer rate the business already tracks, which turns the result into revenue terms for the decision.

Illustrative example — not a real result

A clinic scheduling software firm tests instant booking

Brightquay, an invented shift-scheduling software company for clinics, splits 1,000 demo requests over fourteen weeks: 500 see a booking calendar after the form and 500 receive the usual callback. After a three-week window, demos held per request are 52% for the calendar arm and 41% for the callback arm, with a 95% interval for the difference of roughly 5 to 17 points. No-shows among bookings are 18% against 9%, and the sales-qualified rate of held demos is 62% against 66%, inside the agreed limit. But out-of-profile companies rise from 6% to 11% of held demos. Brightquay adopts instant booking with one qualifying question before the calendar.

Pitfalls

Reading bookings rather than held demos is the first pitfall. The calendar arm almost always books more, and the gap shrinks or disappears once no-shows are counted. Calendar availability differing by week is the second. If the calendar shows few slots in a week when reps are at an event, requesters in that arm see a poorer offer, while the callback arm is unaffected; keep slot availability broadly steady and log the weeks when it was not. Reps calling the booking arm too is the third. If they phone everyone who booked, the calendar arm receives the calendar and the callback, and the test no longer compares the two mechanisms.

Time zones matter for teams that sell across regions: show slots in the requester's local time, and make sure the callback arm is not disadvantaged by calls placed outside the requester's working hours.

Common mistakes

Scheduling tests are lost in the gap between booking and holding. The lead response time test changes how fast the callback happens rather than replacing it, and the experiment library lists every design.

  • Declaring the calendar a winner on bookings. Count demos that were held.
  • Changing the form at the same time. Only the step after submission should differ.
  • Letting reps call both arms. Agree contact rules for each arm before launch.
  • Running through weeks with thin availability. Empty calendars penalise the booking arm.
  • Reading before callback demos have happened. The callback arm's demos arrive later.
  • Ignoring who books. Out-of-profile bookings cost sales time even when they are held.

Frequently asked questions

Why read demos held rather than demos booked?

Because instant booking makes booking easy, including for people who are not committed. Some of those bookings never take place. Demos held count only conversations that happened, which is what sales time is for and the first step towards pipeline, so the metric cannot be inflated by easy bookings that end in no-shows.

Does the form itself change in this test?

No. The form, its fields and its button stay the same in both arms; only the step after submission differs. If the form needs work, test it separately first, for example with a lead form length test, so this test measures the booking mechanism alone.

What if sales still wants to call people who booked?

Agree before launch what contact the booking arm receives: typically an automated confirmation and reminder, and a call only if the booking is missed. If reps also call everyone who booked, the booking arm gets both treatments and the comparison is lost.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Also related

Sources

  1. Website Forms Usability: Top 10 Recommendations — Nielsen Norman Group

Related