Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Marketing Data & Analytics Library › Working with Benchmarks

How to Use Industry Benchmarks

How to use published industry benchmarks properly: reading the methodology, sample and date, and matching the definition to yours.
Guideintermediate10 min readUpdated

What a benchmark can and cannot tell you

A benchmark is a summary statistic, usually a mean or a median, of a metric across a sample of businesses over a period. It can tell you roughly where a population sits, which is useful when a metric is new to you and you have no sense of its scale. It can tell you that your figure is far outside a range on a matched definition, which is a reason to check tracking before anything else, because the most common cause of an implausible figure is a measurement fault rather than exceptional performance. It can tell you how a metric differs across industries or channels within the same report, where at least the definitions are consistent, and it can give a board or a client a frame of reference when they ask for one.

It cannot tell you whether your figure is good. "Good" depends on what the business is trying to do and what the metric costs: a low click-through rate (CTR) on deliberately broad targeting may be the right trade, and a high conversion rate (CVR) on a tiny audience may be a sign of under-investment. A benchmark cannot tell you why a gap exists, because it knows nothing about your product, price, market, season or tracking. It cannot tell you what to change, and it cannot tell you what will happen if you do. Most importantly, it cannot be compared with your figure at all until the definition has been matched, which is the step most comparisons skip. The benchmarks hub describes where published figures come from; the rest of this guide is what to do with one once it arrives.

Reading the methodology

The methodology section is the benchmark. The headline figure is a summary of it, and a summary read without its method is a number with no meaning. Most reports do have a methodology note, often short and near the end; the checks below are what to look for in it, and a report that answers none of them should not be quoted at all.

What to checkWhy it matters
Sample size and compositionA median of a few dozen accounts in one country is a different claim from one across thousands in many; composition decides whether the sample resembles your business at all
How the sample was selectedVendor customers, agency clients and survey respondents are all self-selected in ways that bias the figure, usually upwards for performance metrics
Period coveredA figure measured across a seasonal peak, a platform change or a market shock describes that time, not this one; check the dates against your own calendar
Metric definitionNumerator, denominator and grain: clicks or sessions or users, which conversions count, which attribution window applies; this is the check that most often ends a comparison
Collection methodMeasured in the vendor's tool, exported from accounts, or typed into a survey; each has its own error and the survey has the most
AggregationMean or median, weighted by spend or by account; a mean across accounts of wildly different size says little about any of them
Segmentation availableFigures by industry, channel, region or business size are more useful than one overall figure, if the segments are large enough to mean anything
Treatment of outliers and missing dataWhether extreme accounts were removed, and how accounts with no conversions were handled, changes a median considerably
Publisher's interestA vendor benchmarking its own product has a reason to present it well; this does not make the figure wrong, but it raises the standard of evidence
Date of publication and of dataReports are republished, and compilations copy figures years after the data was collected; the data date is the one that matters

Write the answers down next to the figure. A benchmark that travels through a company without its methodology becomes folklore within a quarter, and the person who quoted it first is usually blamed when the comparison turns out to be meaningless.

Matching definitions

The same metric name hides different quantities, and the differences are large enough to reverse a comparison. A conversion rate can divide conversions by clicks, sessions or users; it can count every conversion event or one per session; it can include or exclude returns, cancellations and test orders; it can use the ad platform's conversion count, which applies the platform's attribution window and modelling, or the analytics tool's, which applies a different window and a different consent treatment. Return on ad spend (ROAS) can use attributed revenue from the platform, last-click revenue from analytics or order revenue from the finance system, before or after discounts, tax and shipping. The platform versus analytics discrepancies guide explains why those sources disagree even inside one business; a benchmark adds another business's choices on top.

The method is to write both definitions side by side: the publisher's, as far as the methodology states it, and your own, as your KPI dictionary states it. Where they match, compare. Where they differ in a way you can correct for, recompute your own figure on the publisher's definition for the purpose of the comparison, and label it as such. Where they differ in a way you cannot correct for, such as a different attribution model or an unknown treatment of returns, record that the comparison is approximate and say in which direction the mismatch pushes it. Where the publisher does not state the definition at all, the comparison is not possible, and the honest report says so. A KPI whose definition is written down in your own dictionary is the prerequisite for any of this, which is one of the reasons the dictionary exists.

Illustrative example — not a real result

A software company matching a trial conversion benchmark

A software company's growth lead is sent an industry survey that reports a median visitor-to-trial conversion rate for companies like hers, and the figure is comfortably above her own dashboard's. Before replying, she writes the two definitions out. The survey asked respondents for "website conversion rate to free trial" with no further guidance; her dashboard divides trial activations confirmed in the product by unique users across the whole site, including documentation and support pages that a trial could never come from. She recomputes on a narrower denominator, visitors to the pricing and sign-up pages, and on trial sign-ups rather than activations, which is the looser definition most survey respondents would plausibly have used. On that basis her figure sits within the survey's reported range. She reports both figures with their definitions, notes that the survey's self-reported sample probably leans optimistic, and proposes that the team keep the strict internal definition for targets and use the looser one only for this kind of external comparison.

Compare against yourself first

Before any external comparison, compare the metric with its own history. Your past figures share your definitions, your tracking, your market and your product; a change against them means something, where a gap against an external average may mean nothing. Build the history as a distribution rather than a single baseline: the range the metric has moved within over a year, by month, by channel and by the other segments decisions are made on. Note the conditions behind each movement, launches, price changes, budget shifts, tracking fixes, seasonal peaks, so that a reader later can separate caused movement from noise. A metric with a wide normal range cannot be judged on one month's value against anything, internal or external, and knowing the range is the first defence against over-reading a benchmark. The control charts framework shows how to draw that range as limits around a centre line and set alert rules that respect seasonal peaks. Reading your own history is itself a period comparison, and comparing periods in marketing reports covers aligning weekdays and holidays and handling tracking breaks before the history is trusted.

Segment before comparing. An overall conversion rate blends channels with very different intent, and a benchmark split by channel compared with your blended figure is a comparison of unlike things even when the definitions match. Compare paid search with paid search, new customers with new customers, one market with that market. Where the benchmark offers no segmentation, compare it with the segment of your business that most resembles the benchmark's sample, and say that you did. The metrics layer guide and the dictionary behind it give the segments and definitions; the history gives the range; only then is there a stable figure to hold against someone else's. The core metrics guide is the reference for what each metric measures when deciding which of your own figures corresponds.

From benchmark to hypothesis

A gap between your figure and a matched benchmark is not a finding. It is a prompt to form a hypothesis about why the gap exists and what would change it, which is a different and more useful thing. The reasoning runs in steps. First, exclude measurement: is the gap explained by tracking, consent, attribution or definition? A large share of apparent gaps are, and the fix is in the data, not the marketing. Second, exclude strategy: is the gap a consequence of a deliberate choice, such as broad targeting that trades click-through for reach, or a premium price that trades conversion for margin? A gap that follows from a choice the business stands by is not a problem. Third, for whatever gap remains, list the plausible causes in your own context, pick the one with the most evidence behind it, and write it as a testable statement: if we change this, then that metric should move in this direction, for this reason.

The writing hypotheses guide covers the form of a good hypothesis and the evidence it should rest on; the benchmark's role is to have suggested that something is worth looking at, not to have decided what. A benchmark that produces a test has done all it can. A benchmark that produces a target has usually done harm, because the target carries none of the reasoning and all of the mismatch.

Reporting benchmarks responsibly

When a comparison reaches people who will act on it, the report must carry what the headline cannot. State the source, the data date, the sample and the definition in the same breath as the figure, so that nobody can quote the figure without them. Show your own figure on the matched definition, and where you had to recompute it, show both versions and say which is used for targets. Show the range of your own history alongside the single external value, so that the comparison is of a distribution with a point rather than a point with a point. Say what the gap could be caused by, in order of likelihood, and what you propose to test; a benchmark comparison that ends in a proposed action is a report, and one that ends in a verdict is a complaint.

Refuse the comparisons that cannot be made. If the definition is unknown, say so and decline to compare, however much the room wants a number. If the sample is clearly unlike your business, say so and offer the internal comparison instead. The credibility of every future comparison depends on the ones you declined to make. Finally, keep a record: the benchmark, its methodology notes, the matched definition and the date you used it, so that when the same figure reappears in a deck a year later, somebody can check whether it was ever comparable and whether it still is.

Common mistakes

  • Quoting the headline without the method. The methodology is the benchmark; the figure is a summary of it.
  • Comparing unmatched definitions. Conversions per click and conversions per session are different quantities with one name.
  • Setting targets from external averages. Targets come from your own baseline and range; benchmarks argue for ambition, not for the number.
  • Comparing a blended figure with a segmented one. Match the segment before matching the value.
  • Treating a gap as a finding. It is a prompt for a hypothesis, and often the hypothesis is "our tracking differs".
  • Forgetting the date. A benchmark describes its period; compilations copy figures long after the data aged.
  • Ignoring the publisher's interest. Vendor benchmarks are not wrong by default, but they deserve a higher standard of evidence.
  • Declining to decline. When no fair comparison exists, saying so is the responsible report.

Frequently asked questions

Should I set targets from industry benchmarks?

Not directly. A benchmark inherits the publisher's sample and definitions, and a target set from it inherits every mismatch with your own data. Set targets from your own baseline and its range, and use a benchmark only as one argument for how ambitious that target should be, after the definitions have been matched.

What if my business has no history to compare against?

Then a benchmark is a sanity check rather than a judgement. Use it to decide whether an early figure is wildly out of range on a matched definition, which would suggest a tracking or setup fault, and start recording your own history immediately so that within a few months the internal comparison can take over.

Why do two benchmark reports give such different figures for the same metric?

Because they measure different samples with different definitions over different periods, and most of the time the headline table does not say so. A conversion rate on clicks from one vendor's customers and one on sessions from a survey of self-reporting marketers are different quantities with the same name. The methodology section of each report is where the difference is visible.

Terms used on this page

Frameworks

Where this page is used

Part of these learning paths

Also related

Related