First, put the agency's number and the business's number side by side
Ask the agency for its headline result for a closed month, by platform: conversions, conversion value and return on ad spend (ROASRevenue generated per unit of ad spend.). Then take the same month from the business's own record: orders and revenue from the order system, or new opportunities and closed deals from the CRM. Put them in one table with a row per platform and a total. If the platform conversions added together exceed the orders that exist, part of the gap is counting, and it is the third cause below.
Next, check that the two columns cover the same thing. Are refunds, repeat customers and offline sales in or out? Is the agency counting a lead, a sign-up or a sale? Is the period the same, given that platforms report a conversion against the date of the ad interaction rather than the day it happened? Write down each difference you find. Most first meetings on this symptom end with a list of definition gaps rather than a verdict on the agency, and that list is the diagnosis.
Finally, look at trend rather than level. If the agency's figures rose while sales stayed flat over several months, the work is either claiming sales that were coming anyway or producing results the ledger does not count. If both fell together, this is a different problem, and the results down across all channels page is the better starting point. When the agency offers its own reason for a movement, the checks in explaining a change in a marketing metric test whether that reason holds up in data it did not come from.
Diagnosis table
Diagnosis
| Symptom | Check | Likely cause | Fix |
|---|---|---|---|
| Report leads with clicks, conversions and platform return; no line for orders or revenue | Which metrics the monthly report shows and where each comes from | Reports built on platform metrics, not the business outcome | Put the business outcome at the top of every report |
| Nobody can say what commercial result the agency is responsible for | The brief and scope: is a business objective and measure written there? | Brief and scorecard never tied to a business objective | Rewrite the objective and measure into the scope |
| Platform conversions added together exceed real orders | Platform totals against the order or CRM count for one closed month | Overlapping, over-claimed platform conversions | Report each platform separately against one deduplicated total |
| Return keeps rising while new customers do not | Share of spend on retargeting lists and brand search, and its trend | Spend drifting to people already intending to buy | Cap capture spend and fund reaching new people |
| Recommendations always involve more budget or more deliverables | How the fee is calculated and what it rises with | Fee model rewarding spend or activity | Align part of the fee with an outcome both sides trust |
| Leads are reported but nobody knows which became customers | Whether lead stages and outcomes are recorded in the CRM | Leads never closed out, so sales never meet the report | Record outcomes and send them back to reporting |
Fix, verify and prevent each cause
Reports built on platform metrics
An agency report usually starts with what the agency can see: impressions, clicks, cost per click, platform conversions and return. Those are links in a chain, as the core metrics guide sets out, and the business outcome sits at the far end of it, often outside the platform's view. A report that stops at the platform links can improve every month without the business noticing any difference.
Fix: ask for the report to open with the business outcome for the period, taken from your own system, and to place the platform metrics underneath as the explanation.
Verify: at the next monthly review, the first question asked is about the outcome line, and the discussion of platform metrics refers back to it.
Prevent: make the outcome line a fixed part of the report template and refuse a report without it. A conversion optimisation agency's version of the problem is test wins quoted as single figures and added together, which the guide to hiring and evaluating a CRO agency replaces with ranges, every test listed and the changes that actually shipped.
A brief and scorecard with no business objective
If the brief asked for leads, traffic or "awareness" without saying which commercial result they serve, the agency will optimise what it was asked for. Kaushik's measurement model starts from the business objective and derives goals, measures and targets from it; a brief written the other way round leaves the agency nothing to aim at but the platform.
Fix: rewrite the objective in the scope as a business result with one primary measure and its source, following the briefing section of selecting, briefing and reviewing agencies.
Verify: ask the agency to play the objective back in its own words and to name the measure it will be judged on; both answers should match yours.
Prevent: do not sign a brief or scope whose objective cannot be traced to a line in the business plan.
Several platforms claiming the same sales
Each platform counts a conversion when it can connect the sale to one of its own ads within its own window. Google documents that its conversion tracking counts actions that follow an interaction with an ad, and Meta documents its own click and view windows in attribution settings. A buyer who saw a social ad and later clicked a search ad can be counted by both.
Fix: stop adding platform figures together and report them beside a single deduplicated total, as the guide to platform attribution versus analytics explains.
Verify: the report now shows platform figures, the analytics or CRM figure and the explained gap for each channel.
Prevent: write into the scope that platform totals are never summed and that the reference figure for sales is the business's own system.
Budget drifting toward people who were already buying
Retargeting lists and brand search reach people who already know the business. They report low cost and high return because many of those people would have bought anyway, which is why budget managed on reported return drifts toward them. The prospecting versus remarketing comparison explains the mechanism and how holdout tests measure what that spend really adds.
Fix: ask for spend split by audience type, set a cap on retargeting and brand spend, and fund work that reaches new people, judged on new customers.
Verify: new customers as a share of sales rise over the following quarter, even if reported return falls.
Prevent: review the split of spend between new and existing audiences at every monthly review, not only the blended return.
A fee that grows with spend or activity
A commission on media spend or a retainer priced by deliverables gives the agency a reason to recommend more of both, whatever the result. That does not make the advice wrong, but it means nobody in the relationship is paid to notice when sales are flat. The in-house vs agency comparison argues that measurement should sit with the side that is not judged by it.
Fix: keep the measure in-house, and when the contract is next reviewed, discuss tying part of the fee to an outcome both sides trust and the agency can influence.
Verify: recommendations in the next review include reallocating or cutting spend where the outcome does not justify it, not only adding to it.
Prevent: ask, for every budget increase, what result would make the agency recommend reversing it.
Leads that are never closed out
In a lead-based business, the agency's report ends at the form and the sale happens weeks later in the CRM. If sales never marks leads as qualified, won or lost, nobody can say which campaigns produced customers, and the agency keeps optimising toward whatever produces form fills. The guide to connecting the CRM to ads and analytics covers capturing the click identifiers, recording outcomes and sending them back.
Fix: agree the lead stages with sales, make outcome recording part of the sales routine, and add won deals by campaign to the report.
Verify: the report shows cost per customer by campaign, and the share of leads without an outcome falls month by month.
Prevent: check the share of unresolved leads at every review and treat a rising share as a measurement fault.
A prevention routine for agency reporting
The causes above share one root: the report describes the platform rather than the business, and nothing in the relationship forces the two together. A short routine keeps them together. Every month, start the review with the business outcome from your own system, then the agency's platform figures beside it with the gap explained, and only then the platform detail. Hold one measure (KPIA metric tied directly to an objective.) per objective, written in the scope, and keep its definition and its source in-house. Every quarter, look at three ratios over time: platform conversions against real orders, new customers against all customers, and the share of spend on audiences that already know the business. A rising first ratio means more over-claiming; a falling second or a rising third means the work is harvesting rather than growing. Twice a year, run or commission a holdout test on the largest capture line, so that the claim of incremental value rests on evidence rather than on platform AttributionAssigning conversion credit to touchpoints.. Keep a written log of every change to definitions, windows and campaign structure, so that a jump in the agency's figures can be traced to a rule change before anyone celebrates it. The agency onboarding checklist sets that log, the measures and the baseline up in the first weeks, before the agency changes anything. When the agency changes, the same routine protects the handover, and the results dropped after changing agency page covers what to check when a new team takes over. If the measurement gaps are closed and the agency still falls short on the business's own figures, the tutorial on turning round an underperforming agency relationship sets out a written improvement plan before any decision to replace it. Where the agency is hired to run paid media across several platforms, the guide to hiring and evaluating a performance marketing agency sets out the records to keep in the business's name, the written definitions to settle in the brief and the exit to ask about before any proposal is weighed.
A furniture retailer reads its agency report against the ledger
An online furniture retailer paid its agency a commission on a monthly media budget of about £60,000. The agency's report for the quarter showed platform return rising from 4.1 to 5.6 and platform conversions up 38%. The order system showed revenue flat and new customers down 9%. Side by side, the search and social platforms together claimed 2,140 purchases in the last month against 1,520 real orders. Spend on retargeting and brand search had risen from 30% to 52% of the budget over two quarters. The retailer rewrote the scope around new-customer revenue measured in the order system, capped retargeting and brand at 30%, and moved the commission to a fixed fee plus a bonus on new customers. Reported return fell to 3.8 the next quarter; new customers rose 14%.
Common mistakes
Celebrating a platform return without asking what the ledger says. Adding platform conversions together. Letting the agency define the measure it is judged on. Cutting the agency before checking whether the brief ever named a business objective. Reading a fall in reported return after a fix as failure, when it is the counting becoming honest.
