Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Marketing Analytics › Business Intelligence for Marketing

Explaining a Change in a Marketing Metric

A method for explaining why a marketing metric moved: rule out data changes, split it into parts, separate mix from rate, find the segment, test the story.
Guideintermediate12 min readUpdated

Start by doubting the change

Before asking why a number moved, ask whether it did. Some of the movements that reach a monthly review are produced by the measurement rather than the market, and an explanation built on one of those wastes everyone's time and sometimes a budget. Work through four checks before any analysis.

CheckWhat to look forWhere to look
TrackingA tag that stopped firing or fired twice, a consent change, a new checkout domain, bot traffic, a filter added or removedThe data quality routine and the business system's own count of orders or leads
DefinitionA metric version change, a new attribution setting, a changed filter in the dashboard, a different source tableThe metric's entry in the metrics layer and the dashboard's change log
CalendarPeriods with different numbers of weekdays, a holiday that moved, a partial period, recent days still collecting conversionsThe rules in comparing periods
VolumeA rate built on a handful of conversions, which moves this much routinelyThe counts behind the rate, read as the small numbers guide describes

The fastest single check is the business system. If the shop's back office, the CRM or the booking system shows the same movement in orders, leads or bookings, the change is probably real. If the business count held steady while the analytics figure moved, the measurement is the first suspect, and the platform versus analytics discrepancies guide explains why two systems can disagree without either being broken. For a conversion rate that rose with no change anyone can name, the diagnosis for a conversion rate that jumped overnight applies these checks to double counting, spam entries, lost visits and a shift in who arrives.

Keep a running list of everything that changed in measurement and on the site, with dates: tag container versions, releases, consent banner changes, new campaigns, price changes. That list turns this step from detective work into a lookup, and it is the same list the event-matching step below needs.

Decompose the metric

Most marketing metrics are built from other metrics by arithmetic, and the first question is which part of the arithmetic moved. Revenue is sessions multiplied by conversion rate (CVR) multiplied by average order value (AOV). Cost per acquisition (CPA) is cost per click divided by conversion rate. Leads are the sum of leads from each channel. Each of those parts can be broken down again, and the KPI trees guide shows how to build the full tree once so that every KPI on a dashboard has its parents and children written down.

Read the decomposition for the two periods side by side and note, for each part, its direction and its size. Three patterns are common:

  • One part moved and the rest held. Revenue fell because order value fell while traffic and conversion rate were flat. The question narrows to that part.
  • Parts moved in opposite directions. Sessions rose and conversion rate fell, leaving orders flat. This usually means new traffic arrived that converts differently, which is a mix question for the next step.
  • Everything moved a little. A small change in every part, in the same direction, often points to a calendar or seasonal cause rather than a single event, or to ordinary variation.

For multiplicative metrics, compare the relative change of each part rather than the absolute change, because the relative changes add up, approximately, to the relative change in the total. That is what lets a team say that most of a revenue fall came from order value and the rest from conversion rate, instead of listing three movements and leaving the reader to weigh them.

Mix and rate

The most common wrong explanation in marketing reporting comes from a blended rate. A site's conversion rate is an average of the conversion rates of every segment of its traffic, weighted by each segment's share of sessions. It can move for two different reasons: the rates inside the segments changed, or the shares changed. The second is a mix effect, and it moves the blended rate even when no segment converts any differently.

A mix effect is not a problem in itself. A successful campaign that brings a large volume of colder traffic lowers the blended rate while adding orders. Treating that fall as a conversion problem leads to the wrong fix: a checkout redesign for traffic that was never going to convert at the old rate, or a cut to the campaign that was adding orders.

The decomposition is arithmetic and can be done in a spreadsheet:

  1. Choose segments that are defined identically in both periods: device by channel is the usual start.
  2. For each segment, record its share of sessions and its conversion rate in each period.
  3. Mix effect for a segment: the change in its share, multiplied by its conversion rate in the earlier period. Add these across segments.
  4. Rate effect for a segment: its share in the later period, multiplied by the change in its conversion rate. Add these across segments.
  5. The two totals add up to the change in the blended rate. Whichever is larger is where the explanation lies.

A useful companion number is the reweighted rate: the later period's segment rates weighted by the earlier period's shares. If it sits close to the earlier blended rate, the segments behaved as before and the traffic moved between them.

Illustrative example — not a real result

Puddlecraft Rainwear's conversion rate falls while its segments hold

Puddlecraft Rainwear is an invented online rainwear brand. Its four-week report shows site conversion rate down from 2.59% to 2.11%, and the weekly meeting starts discussing a checkout review. The analyst first checks the order system, which shows orders up from 3,620 to 4,010, then splits the two periods by device and channel:

SegmentShare of sessions, beforeShare, afterConversion rate, beforeRate, afterMix effect (points)Rate effect (points)
Desktop, search28.6%20.0%3.50%3.48%−0.300−0.004
Mobile, search42.9%32.1%2.40%2.38%−0.258−0.006
Mobile, social14.3%36.8%0.90%0.87%+0.203−0.011
Email and direct14.3%11.1%3.00%2.99%−0.097−0.002
Total140,000 sessions190,000 sessions2.59%2.11%−0.452−0.023

Of the 0.475-point fall, 0.452 points is mix and 0.023 is rate. Mobile social sessions rose from 20,000 to 70,000 when the social team launched a video campaign for a new jacket, and that traffic converts at well under half the site average. Reweighted to the earlier period's shares, the later rates give 2.56%, close to the original 2.59%. The analyst tests the story on data it did not come from: the campaign's own landing page shows the same low rate in its first and last week, the device-only breakdown that had worried the meeting (mobile down from 2.03% to 1.57%) is itself mix within mobile, and the week after the campaign paused the blended rate returns to 2.52%. The write-up recommends judging the campaign on its cost per order, not on the site rate, and the checkout review is dropped.

The example also shows why a breakdown by one dimension is not enough. Mobile's rate fell sharply because mobile traffic itself became more social; only the combination of device and channel showed segments holding steady.

Finding the segment that moved

When the rate effect is the larger part, or when the metric is a count rather than a rate, the next job is to find where the change sits. Segmentation is the tool, and the order of work matters more than the number of breakdowns.

Go in order of size. A segment can only move the total in proportion to its volume, so start with the dimensions whose largest values carry most of the metric: channel, device, market, landing page group, new against returning, product category. For each, list every value's contribution to the change in the total, in the metric's own units, and sort by it. The contributions add up to the total change, so the top two or three rows usually account for most of it.

One dimension, then two. Read each dimension on its own first, then cross the two that matter most. The mix trap applies at every level: a change in one dimension can be produced by a shift in another. A breakdown in GA4 explorations or a grouped query in the warehouse both work; what matters is that the same segment definitions are applied to both periods.

Stop at something actionable. The drill ends when the change is concentrated in a segment that a person can act on or investigate: one campaign, one browser, one product line, one market. A change that stays spread evenly across every segment after several cuts is itself a finding; it points to something that affects everyone, such as price, a sitewide release, Seasonality or tracking.

Watch the small segments. A segment with very few conversions can show a dramatic rate change that contributes almost nothing to the total. Sorting by contribution rather than by percentage change keeps those from leading the investigation.

Matching the change to an event

A segment tells you where; an event tells you why. List what happened around the date the change began, in and outside the business: site releases and tag changes, campaigns launched, paused or rebudgeted, price and offer changes, stock outs, delivery changes, competitor moves you know of, platform changes in bidding or attribution, and the calendar. The running list from the first step is the starting point.

Three tests make a match credible rather than coincidental:

  • Timing. The change starts when the event starts, at the daily or hourly level, not merely in the same month.
  • Location. The change sits in the segment the event touched. A pricing change on one product line should move that line, not every category.
  • Dose. Where the event was larger, the change is larger: the markets with the deepest discount moved most, the campaign with the most spend added the most cold traffic.

When the first match is found, ask why again. A fall in a channel's conversion rate traced to a new campaign is a location; the cause may be the campaign's audience, its landing page or a broken tracking parameter. The five whys framework gives the discipline of asking until the answer is something a person can change, and its limits section is a reminder that a single chain can miss causes that act together.

Testing the explanation

An explanation found by searching the data will always fit the data it was found in. The test is whether it predicts something in data it did not come from. Write down, before looking, what else must be true if the explanation is right, and then look.

If the explanation isIt also predictsCheck
A mix shift towards colder trafficSegment rates roughly unchanged; the reweighted rate close to the old oneThe decomposition and the reweighted rate
A broken step in the journeyThe fall concentrated at one step and one template, browser or deviceStep-by-step rates for the affected segment
A price or offer changeOrder value or conversion moving in the affected products onlyProduct-level figures, before and after
A tracking faultThe business system disagreeing with analytics from the same dateOrders, leads or bookings in the source system
SeasonalityThe same shape in the same weeks of earlier yearsThe business's own history, aligned by week
A campaign launch or pauseThe change starting and stopping with the campaignDaily figures around both dates

Two further tests are worth the effort when the decision is large. Look for a reversal: if the event ended, did the metric return? And look for a comparison group: a market, product line or segment the event did not touch should not have moved. Where neither is available, say so; the explanation is then a hypothesis with supporting evidence, not a finding. Where a team needs proof that an action caused a result, a controlled test is the tool, and the conversion rate troubleshooting page shows how fixes for named causes are verified.

Writing it up

The write-up is short and structured, so that a reader can see what was checked, what was found and how sure the analyst is. The table below works as a template; its fields fit inside the commentary of the monthly marketing report template or stand alone as a note.

FieldWhat to write
What movedThe metric, its definition version, the size of the change and the two periods compared
Is it realThe tracking, definition, calendar and volume checks made, and their results
Which partThe component of the decomposition that moved, and by how much in the metric's units
Mix or rateThe split between the two, and the reweighted figure if it was computed
WhereThe segment or segments carrying most of the change, with their contribution
WhyThe event matched, with the timing, location and dose evidence
Tested howWhat the explanation predicted elsewhere and what that data showed
Still uncertainWhat could not be ruled out, and what data would settle it
What changesThe decision or action proposed, or "no action" with the reason
WatchWhat will be checked next period, and by whom

The "still uncertain" field is the one most often left empty and the one that most protects the analyst's credibility. A note that says the explanation accounts for most of the change and names what is left unexplained will survive the next period; a note that claims certainty may not. The BI hub places this kind of analysis among the other jobs a reporting stack has to support. The same checks apply when a marketer writes up their own part in a result, which the guide to showing your contribution to marketing results honestly covers from the individual's side.

Common mistakes

  • Explaining before checking. A tracking change produces a perfectly explicable-looking movement; clear the data, definition and calendar first.
  • Treating a blended rate fall as a conversion problem. Run the mix and rate split before proposing a fix to pages or checkout.
  • Breaking down by one dimension and stopping. A single breakdown can still hide a mix change; cross the two dimensions that matter most.
  • Sorting segments by percentage change. Small segments dominate that list; sort by contribution to the total.
  • Accepting the first event that lines up. Check timing, location and dose, and ask why again.
  • Testing the story on the data that produced it. Write down what else must be true and look at data the search did not use.
  • Comparing unlike periods. Weekday mix, moving holidays and late conversions produce changes that no segment explains.
  • Leaving out what is uncertain. The next period will reveal it anyway.

Frequently asked questions

How can I tell whether a fall in conversion rate is a mix effect or a real change?

Recalculate this period's rate as if traffic had arrived in last period's proportions: take each segment's new conversion rate, weight it by that segment's old share of sessions, and add the results. If the reweighted rate sits close to last period's rate, the segments converted much as before and the blended figure moved because the traffic moved between them. If the reweighted rate falls as far as the reported one, the rates inside the segments changed and the question becomes which segment and why. The segments have to be defined the same way in both periods for the comparison to mean anything.

Which segments should I look at first?

Start with the dimensions that carry the most volume and the ones whose share changed most between the two periods, because a segment can only move the total in proportion to its size. Device, channel, landing page group, new against returning visitors and market cover most marketing questions. Look at one dimension at a time first, then at the combination of the two that matter most, since a single breakdown can still hide a mix change inside it. Stop drilling when the change is concentrated in a segment someone can act on.

What should I report if no explanation survives testing?

Report the movement, the checks that ruled out a data problem, the explanations that were tested and why each failed, and a plan to watch the metric. An honest unexplained change is more useful than a confident story that the next period contradicts, and it keeps the team from acting on a cause that does not exist. Often the movement is ordinary variation, which the guide to reading small numbers helps confirm, or it explains itself once another period of data arrives.

Terms used on this page

Frameworks

Where this page is used

Related