Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Interactive Tools › Conversion Rate Calculator

Conversion Rate Calculator for Email Flows

Using the conversion rate calculator for automated emails: orders over flow entrants in a fixed window, run again for a held-out group to see the gain.
Tutorialintermediate7 min readUpdated

Why email flows change the calculator's inputs

The default page pictures website traffic: sessions arrive, some convert, and the rate describes the site. A flow is triggered by a person's own action, such as leaving a basket, placing a first order or asking to hear when a product returns, so everyone it reaches has already shown intent. Many of them would buy without a single email. The email metrics and testing guide makes this the central problem of flow reporting: a flow's report claims every order from anyone it reached, so the better it targets, the more it overstates itself.

That changes the denominator first. Site visits mix flow recipients with everyone else, and email clicks leave out the entrants who bought without clicking, so neither describes the group the flow acted on. The only fair base is the set of people who entered, counted at the moment of entry. The numerator needs a boundary too: orders within a window agreed in advance, because an open-ended window lets the flow claim purchases made weeks later for unrelated reasons.

The uplift input changes most. On the default page it is a hopeful dial. For a flow it becomes a measured figure: the lift of the receiving entrants' rate over a Holdout Group chosen at random at entry, which is the only way to see the flow's Incrementality rather than its reach.

The inputs for an email flow

Calculator inputOn the default pageFor an email flowWhere to find it
ConversionsConversions in the periodOrders, or the flow's goal action, placed within the agreed window after the first message, from the shop's own recordsThe order system joined to the flow's entry log
Visitors / sessionsVisitors or sessionsPeople who entered the flow, counted at entry; for a stock-alert flow, the alerts sentThe email platform's entry log for the flow
Relative CVR improvementA hoped-for improvementThe flow's lift over the held-out rate, entered with the held-out counts so the extra conversions show what the flow adds at that sizeReceiving and held-out rates over the same window

The uplift is derived, not chosen. Run the calculator for the receiving entrants and note the rate, then for the held-out entrants and note theirs. Divide the receiving rate by the held-out rate and subtract one; that relative lift is the figure to enter. Leave the held-out counts in place when entering it, and the extra conversions line becomes the orders the flow added for a group the size of the holdout. Multiply it by the ratio of receiving to held-out entrants to get the flow's contribution for the whole period.

Both runs must share the window, the counting rule and the dates. The abandoned cart flow tutorial measures the flow on orders from entrants inside a set period that starts with the first email, and the tutorial on back-in-stock and price-drop alerts uses alerts sent as the base, because an alert goes out only when the product returns or its price falls.

Entering the flow inputs step by step

  1. Split entrants at random before anything is sent

    Set the flow to hold back a random share of everyone who qualifies, at the moment of entry. Held-out entrants still receive service messages and other marketing, just not this flow.

  2. Fix the window and the counting rule

    Choose how long after the first message an order still counts, and whether any order counts or only the item in question. Write both into the flow's notes and keep them unchanged between reports.

  3. Run the receiving group, then the held-out group

    Join the entry log to the order records for each group. Enter the receiving entrants and their orders, note the rate, then enter the held-out entrants and their orders and note that rate.

  4. Enter the lift on the held-out counts

    Work out the relative lift, enter it as the improvement with the held-out counts still in place, and scale the extra conversions up to the receiving group's size.

A worked flow with illustrative inputs

Illustrative example — not a real result

Kruveth Tea reads its abandoned-cart flow against a holdout

Kruveth Tea, an invented online tea shop, runs a three-email abandoned-cart flow. In one month 4,600 shoppers entered it, and a random 10% were held out. The 4,140 receiving entrants placed 306 orders within five days of the opening email, or 7.4%; the 460 held-out entrants placed 25, a rate of 5.4%. The receiving rate is about 36% higher, so Kruveth enters 25 conversions, 460 visitors and a 36% improvement. The calculator shows 5.43% and 9 extra conversions: the flow added about 9 orders for every 460 entrants. With nine receiving entrants for each held-out one, that is about 81 orders for the month, not the 306 the flow's report credited.

With only 25 held-out orders, one month's gap of about two points has a range of roughly −0.3 to 4.2 points, so it cannot yet be told apart from chance. Kruveth reads the holdout quarterly: if the rates hold, three months of entrants narrow the range to roughly 0.7 to 3.2 points, clear of zero.

Conversion Rate Calculator

Open the Conversion Rate Calculator →

Prefilled with the example's inputs: Conversions 25 · Visitors / sessions 460 · Relative CVR improvement 36% → Conversion rate 5.43% · Extra conversions from uplift 9.

All figures above belong to the invented tea shop. Enter the receiving group's counts in the embedded calculator to see the rate the flow's own report would celebrate.

Interpreting the rate per entrant and the held-out gap

Read the conversion rate (CVR) first as a rate per entrant within the stated window, and print the window beside it. The email metrics guide warns that altering the attribution rule shows up as if performance had moved, so a rate that jumps after someone lengthened the window has not improved. Compare each month with the same flow's history under the same rule rather than with any outside figure.

Then read the gap, not the receiving rate. Only the difference between receiving and held-out entrants is what the flow adds; the abandoned cart tutorial flags the share of orders from shoppers who returned before any email as the warning sign of a flow that looks excellent and adds little. The extra conversions output on the held-out counts, scaled to the receiving group, is the contribution to report, and it is the figure to weigh against any incentive the flow carries.

Read results by monthly Cohort of entrants and by branch. The post-purchase flow tutorial reads second orders by monthly cohort of first-time buyers, reviews each branch monthly and reads the holdout quarterly, and the same rhythm suits other flows: entrants from one month form one cohort, judged once their window has closed.

Give back-in-stock alerts a rate of their own and price-drop alerts another, as the alerts tutorial asks, with alerts sent as the base and orders within a stated window as the conversions. Click-through and the alert's Add-to-Cart Rate are diagnostics: a burst of clicks that ends without baskets says more about the product page, the price or sold-out stock than about the message.

Finally, the calculator cannot tell whether a gap is real. A flow that gains only a handful of entrants a month leaves its holdout too thin to read in any single month, and its rate should carry that caveat until enough entrants have accumulated. The email metrics guide accepts holding out for months on a small flow, and the guide to reading small numbers shows how to put a range on each rate and read whether the two ranges overlap before calling a difference. For click-based reading of one-off sends rather than flows, the CTR calculator for email campaigns uses delivered messages as its base, and conversion rates for landing page tests compare two arms of a split test instead.

Pitfalls when flow numbers go into the calculator

  • Dividing flow orders by email clicks or site visits. Clicks drop entrants who bought without clicking and visits mix in everyone else; the base is entrants, counted at entry.
  • Leaving the attribution window open-ended. Without a fixed window the flow keeps collecting orders long after its last message, and every change to the window reads as a change in performance.
  • Crediting the flow with orders from shoppers who returned before any email. They were on their way back; only the held-out comparison removes them.
  • Pooling cart and browse flows in one rate. The two carry different intent and behave differently, so a blended rate hides which one moved. Run each separately.
  • Reading a holdout too small to separate from chance. A gap from a few dozen held-out orders can vanish next month; report it as unresolved until the range clears zero.

Frequently asked questions

Should a flow's conversion rate be worked out from clicks or from entrants?

From entrants. Everyone who qualifies for the flow and enters it is the group the flow is trying to move, and some of them buy without ever clicking. Dividing orders by clicks leaves those buyers in the numerator while dropping their peers from the denominator, which inflates the rate and makes flows with few clicks look better than they are.

How big does the held-out group need to be?

Big enough that the gap between the receiving and held-out rates stands clear of what chance alone could produce. A busy flow can get there with a small share held out; a quiet one may need months, and until then the result should be reported as unresolved rather than as a win.

Does holding people out of a flow cost sales?

A little, in the short run. The held-out entrants miss whatever the flow adds, which is the gap between the two rates times the number held out. That cost buys the only reliable answer to whether the flow, and any incentive inside it, earns its place, which is why many shops keep a small permanent holdout on their busiest flows.

Terms used on this page

Tools to try

Related