Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Marketing Frameworks › The Experimentation Loop and ICE

Experimentation Loop for Lead Generation

Running the experimentation loop for lead generation: testing on lead quality not volume, small samples, lagged outcomes and proxies checked against pipeline.
Objective guideintermediate7 min readUpdated

Why the fast-outcome loop misfires on leads

The loop's standard cycle assumes the outcome arrives during the test and in volume: a click, a sign-up, a purchase. Lead generation breaks both assumptions. The event the test tool sees, a submitted form, is a promise of value rather than value, and the promise varies by variant: a shorter form or a softer offer attracts more people and, often, less committed ones. Meanwhile the event that confirms value, a qualified lead or an opportunity, depends on sales calling the lead, and arrives days or weeks after the test has stopped.

The volume is wrong too. A page that produces many submissions produces far fewer qualified leads and fewer still opportunities, so a test powered for submissions is usually underpowered for anything deeper. Whether the programme should be testing for captured demand at all, or for the demand-creation work that fills the top of the funnel, is a strategic question settled in the lead generation versus demand generation guide before the backlog is built.

Decisions that change for lead generation

Loop stepIn the generic loopIn a lead-generation loopDecision it forces
HypothesisChange, effect on a named metricChange, effect on lead quality as well as volumeWhich stage the hypothesis predicts will move
Primary metricThe conversion the test tool recordsQualified leads or pipeline per visitorWhat counts as qualified, and who decides
Sample and powerPowered for the primary metricPowered for a proxy, with the proxy validated firstWhich proxy is trusted and on what evidence
ReadoutOnce, at the planned sampleTwice: at the sample, and after the sales cycleThe follow-up date, written down at launch
PrioritiseICE on impact, confidence, easeICE plus the cost of a flood of weak leadsHow downside to sales is scored
Test conditionsVariants differ only in the changeSales handling must also be identicalHow routing and follow-up are kept blind

Set the primary metric on qualified leads or pipeline

The hypothesis card should name a quality metric as primary: qualified leads per visitor, or pipeline value per visitor where volume allows. Form completion becomes a guardrail, there to show the change did not break the form, not to declare a winner. The trade-off is speed. Submission rate gives an answer within days; qualified rate gives a better answer later. Choose qualified leads as primary for any change to the offer, form length or audience, and accept the slower result. The lead form length test design is a worked hypothesis built on exactly this metric. The forms guide covers which fields qualify a lead, which is where most form tests start.

Validate proxies before relying on them

Qualified leads are often too few to reach a sensible sample in any reasonable run, and pipeline fewer still. The usual answer is a proxy available at submission: a lead score, a company-size field, a job title band. A proxy is only useful if it predicts the outcome for leads produced by both variants; a shorter form that drops the company-size field makes that proxy unusable for the very test it was meant to serve. Check, on past leads, that the proxy separates leads that progressed from leads that did not. The lead scoring guide covers calibrating a score against outcomes. The sample size and significance guide explains why the underpowered alternative produces false winners. Choose a validated proxy as the decision metric and keep the true outcome as a confirming readout.

Delay the readout and record a follow-up date

Close the test at its planned sample, write the first result on the proxy, and enter a second readout date in the write-up: the date by which most leads from the test will have reached the stage that matters. The cost is a decision that stays provisional for weeks. The alternative is shipping a change that later turns out to have filled the pipeline with leads that never closed. Choose to ship on the first readout only when the guardrails are clean, and to roll back if the second readout disagrees.

Score ideas for the cost of weak leads

ICE rewards changes that are easy and touch many visitors, which in lead generation means removing fields and softening offers: exactly the changes most likely to lower lead quality. Add the cost to sales to the scoring conversation: how many extra calls a weak-lead flood would create, and how quickly the change could be reversed. A simple way is to lower confidence for any idea whose upside is volume and whose risk is quality. Choose to score that risk explicitly rather than hoping the test will reveal it.

Coordinate with sales so follow-up is identical

If sales knows which leads came from the new form, or if one variant's leads route to a different team, the test measures sales behaviour as much as the change. Agree with sales that leads are routed and contacted by the same rules regardless of variant, record the variant only in a field sales does not see, and note any change in sales process during the test. The same rule applies after a price has been sent: the quote follow-up test is a design that keeps the owner's calls identical in both arms and varies only the automated follow-up messages. The LinkedIn targeting and lead gen forms guide is a reminder that native forms route differently and should not be mixed into a website form test.

Decision points in a worked lead-generation test

Illustrative example — not a real result

A shorter demo form at an invented HR-software vendor, Tallyhand

Tallyhand sells HR software to firms of 50 to 500 staff. Its demo-request form has nine fields. Research: a form-analytics review shows 41% of starters abandon at the phone and company-size fields, and sales says a third of demo calls are with firms below 50 staff. Hypothesis: removing phone and company size, and asking size on the booking page instead, will raise qualified demos per visitor, not just submissions.

Loop artefactContent
Primary metricQualified demos per visitor (firm 50 to 500 staff, demo held)
Proxy for the first readoutEmail domain matched to a firm-size database; validated on 1,800 past leads, where matched size predicted qualification 82% of the time
GuardrailsForm completion rate; share of leads with no matched size
ICEImpact 7, confidence 5 (lowered for quality risk), ease 8, score 280
Sample16,000 visitors per variant, about four weeks
First readoutSubmissions 4.1% to 5.6%; proxy-qualified leads per visitor 1.9% to 2.0%, inside the noise
Second readout dateSix weeks after close
Second readoutQualified demos held per visitor 1.4% control, 1.3% variant; pipeline per visitor £38 against £36

Decision: the shorter form produced 37% more submissions and no more qualified pipeline. Sales handled 260 extra leads in the period. The change was not shipped; the learning is that friction was filtering out small firms, and the next research question is how the booking step asks about size.

Read on submissions alone, this test was a clear win. The decisions above turned it into a learning that saved sales a stream of unsuitable calls, and the follow-up date stopped the first readout from becoming the final one.

Measuring the loop: KPIs and cadence for lead tests

For each test, the primary KPI is qualified leads or pipeline per visitor, with form completion rate, the share of leads with a usable proxy and sales capacity as guardrails. For the programme, useful measures are the share of tests decided on a quality metric rather than submissions, the share whose second readout agreed with the first, and the time from test close to final readout. The conversion rate calculator helps size the stage rates each test needs. Review running tests weekly, hold second readouts as a standing item in the monthly review, and revisit proxy validity each quarter, because what predicts a good lead drifts as the product and market change. Treat an A/B Test that moves CVR on the form but not further down as a finding, not a failure.

Where lead-generation tests mislead

Proxies validated on past leads may not hold for a variant that attracts a different audience. Sales capacity limits can make a high-volume variant look worse simply because its leads waited longer for a call. Small samples at deep stages mean many tests end inconclusive, and teams are tempted to read direction from noise. And tests run on paid traffic inherit whatever the bidding algorithm was doing at the time, so a variant can win because the audience shifted beneath it.

Common mistakes

  • Declaring a winner on submissions. More forms is often more of the wrong leads.
  • Using a proxy the change removes. A field dropped by the variant cannot judge the variant.
  • Forgetting the second readout. Without a date in the write-up it rarely happens.
  • Telling sales which leads are from the test. Their behaviour becomes part of the result.
  • Scoring only upside. Ease and reach favour changes that flood sales with weak leads.
  • Mixing native-form leads into a website test. They route and qualify differently.

Frequently asked questions

Can a lead-generation test ever be decided on form submissions alone?

Only when the change cannot plausibly affect who submits, for example a fix to a broken field on one browser. Anything that changes the offer, the length of the form, the wording of the call to action or the audience will change the mix of people who submit, and the submission count then says nothing about whether the business is better off.

What if the test never has enough qualified leads to reach significance?

Then either test bigger changes that would produce effects large enough to detect, pool similar tests into a longer programme, or decide on a validated proxy with the uncertainty stated. What should not happen is quietly switching the primary metric to submissions because they reached significance first.

How long after a lead test ends should the pipeline readout happen?

Long enough for most leads from the test period to have reached the stage being read. For a qualified-lead stage that is often a few weeks; for opportunities or revenue it is the typical sales cycle. Record the date when the test closes, so the second readout is a scheduled event rather than something remembered by chance.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Also related

Sources

  1. How Not To Run an A/B Test — Evan Miller
  2. About conversion tracking — Google Ads Help

Related