Where AI helps analysts
The applications sort by how easily the output can be checked. Query generation is checkable because the query is visible. Anomaly flags are checkable because the underlying report can be opened. Narration is checkable against the table it describes. Forecasts are checkable only in hindsight, which is why they carry the most caveats.
| Application | What the tool does | What makes the result checkable | Main risk |
|---|---|---|---|
| Plain-language querying | Writes a query or builds a report from a question | The query text; the table it ran against; the definition it used | Right answer to a different question |
| Anomaly detection | Flags movements outside normal variation | The underlying metric in the standard report | Flags on tracking breaks or seasonality read as real change |
| Alerts | Notifies when a condition is met | The condition, stated plainly | Thresholds set without reference to normal noise |
| Summarising reports | Turns tables and dashboards into narrative | The table beside the narrative | Confident explanations of causes the data does not show |
| Narrating changes | Explains why a metric moved | The breakdowns that support the explanation | Plausible stories; mistaking correlation for cause |
| Forecasting | Projects a series forward with a range | History the forecast used; its assumptions | Point estimates treated as commitments; shocks unmodelled |
| Data preparation | Cleans, joins and reshapes exports | A row-level comparison with the source | Silent dropped rows and mismatched joins |
| Attribution and modelling | Fits models that allocate credit or estimate effect | The model's inputs, assumptions and validation | Outputs read as truth rather than as one model's estimate |
Two applications are omitted on purpose. Deciding what to measure is a design task, and the measurement-plan framework this guide pairs with covers it; a tool can draft a plan but cannot know the business. And judging whether a KPIA metric tied directly to an objective. is the right one is a strategy question that no amount of querying answers.
Plain-language querying
Asking a question in plain words and getting a table back is the most used application and the one most likely to produce a confident wrong answer. The tool has to translate the question into choices: which table holds the data, what counts as a conversion, which date range, which time zone, whether sessions or users are the unit, how channels are grouped, what is excluded. Every one of those choices is a place where the tool can pick something reasonable that is not what the asker meant, and the table it returns will not say so.
The remedy is to treat the query as the answer and the table as its consequence. If the tool shows the query it ran, read it; if it does not, ask it to restate in plain words what it counted, from where, over what period and with what filters, and compare that to the question. Reading a query is a smaller skill than writing one, and the SQL for marketers guide teaches enough to do it. Where the organisation has a metrics layer, the single place where "conversion", "revenue" and "active customer" are defined, the tool should query that rather than raw tables, so that it inherits the definitions instead of inventing them; the metrics layer and definitions guide explains why that layer exists.
Google Analytics documents a search box in the property that answers questions about the data in plain language and returns the relevant report or figure; the help centre is the record of what it can and cannot answer. The same rule applies to it as to any tool: confirm the dimension, metric and date range it chose. The GA4Google's event-based analytics platform. reports it points to are the place to check.
A query without digits is still worth asking the tool to produce, because the shape of the query is the check. A request for "orders by first-touch channel last month" should produce something like the following, and a marketer can read it even without being able to write it.
select first_touch_channel, count(distinct order_id) as orders
from orders
where order_date >= :period_start
and order_date < :period_end
group by first_touch_channel
order by orders desc;
If the tool instead counted sessions, used last-touch channel or set the period boundaries to include the current month, the table would look just as tidy and be wrong.
Anomaly detection and alerts
Anomaly detection watches a metric and flags movements outside what its history would predict. It is useful because people do not look at every metric every day, and the first sign of a broken tag, a runaway campaign, a pricing error or a bot is usually a number that moved when nothing was supposed to change. Google Analytics documents automated insights that surface unusual changes in the property's data and custom insights that notify when a condition the user sets is met; the help centre describes the conditions and the delivery options.
What a flag does not carry is a cause. A metric can move because demand moved, because a campaign launched, because a tag broke, because a holiday fell differently, because a bot arrived or because a filter changed. Most flags in a marketing property are not real changes in the business, and the discipline is to confirm every flag in the underlying report and name the cause before anyone reacts. The data quality guide describes the checks that separate tracking problems from real movement, and a flag is often the first step of that procedure rather than a result.
Alerts are anomaly detection with a condition the team sets, and the condition is where they go wrong. A threshold set at a round number without reference to normal variation fires constantly on a noisy metric and never on a stable one. Better conditions are relative, describe a sustained change rather than a single day, and are set after looking at how much the metric usually moves. The set of alerts also needs an owner and a review, because alerts nobody acts on train the team to ignore them. A small set of alerts on the metrics that would cost the most if they broke, each with a named responder, is worth more than a comprehensive set that is muted within a month.
Summarising and narrating reports
Turning a dashboard into prose is a real saving when the report is read by people who want the headline and would not read the table, and it carries a specific risk: narrative explains, and the data in a report usually does not. A tool asked to summarise a weekly performance report will describe what moved, which is verifiable, and will tend to say why, which usually is not. "Conversions fell because of lower paid social spend" is a story the tool constructed from two numbers that moved together; the table shows the movement, not the cause.
The workable rule is to let the narrative describe and to make a person explain. A summary that says which metrics moved, by how much and against which comparison is checkable line by line against the report, and should be checked that way, with particular attention to direction, period and the comparison used, which are where summaries go wrong. Any sentence in the summary that gives a cause is either removed or marked as a hypothesis for the analyst to confirm with a breakdown. Where the organisation's definitions differ from the tool's assumptions, as with what counts as a lead or a return, the summary should state the definition it used. Qualitative material needs a different discipline from tables: summarising interviews, open survey answers and reviews is covered in using AI for research synthesis. Whatever the material, strip an export to the fields the question needs before it reaches a tool, following the routine for minimising data before prompting.
Narration has one more use, which is to make reports consistent. A tool given the same template and the same report each week produces the same structure, which readers learn to scan. That is a benefit as long as the template itself was designed by someone who knows what the readers need, and reviewed when those needs change. Attribution deserves a special mention: a narrative that says a channel "drove" a share of revenue is reporting an AttributionAssigning conversion credit to touchpoints. model's allocation, and the summary should say which model, because readers will otherwise hear it as fact.
Forecasting with caveats
A forecast projects a series forward from its own history, and marketing series are among the harder ones to project: they are seasonal, they respond to spend the team controls, they are shaped by launches and competitors, and their tracking changes under them. Tools produce forecasts with ranges that look precise, and the range is the honest part. The caveats come in three groups.
History. A forecast is only as good as the series behind it. A metric with a short history, a tracking change part-way through, or a known one-off in the period is not a sound basis, and the forecast should say what it was trained on. Google Analytics documents predictive metrics, such as the likelihood that a user will purchase or churn, that are available only when the property meets eligibility conditions the help centre describes; the same logic applies to any forecast, which is that thin or unstable data produces estimates that should not be relied on.
Assumptions. Every forecast assumes that what drove the past will drive the future. It cannot see a planned launch, a price change, a new competitor, a platform policy change or a shift in tracking. A forecast used for planning should carry a list of what it assumes stays the same, and the plan should state what the team intends to change.
Use. Forecasts are inputs, not commitments. The useful practice is to record each forecast with its range and assumptions, compare it with what happened, and let that record decide how much weight forecasts get. Teams that do this find that forecasts of stable series earn trust and forecasts of volatile ones do not, which is the right outcome.
A subscription service checks a generated answer
A subscription box company's growth lead asks a tool for "new subscribers by channel in September" and receives a tidy table: paid social 1,140, paid search 860, organic 610, referral 240, email 95. The first-touch total of 2,945 is about 18% higher than the finance team's figure of 2,490 new paying subscribers. The lead asks the tool to restate what it counted. The query had counted sign-up events, including free trials that never converted, had used last-touch channel, and had run on a table in UTC while finance reports in local time, which pulled 31 August's late-evening sign-ups into September. Re-run against the metrics layer's "new paying subscriber" definition with first-touch channel and the local calendar, the table shows 2,490 subscribers, with paid search ahead of paid social, and the planned reallocation of £12,000 towards paid social is dropped. The log records the three mismatches and the team adds "state the definition, touch model and time zone" to its standard query prompt.
Where models mislead
Some failure modes are general to AI tools and some are particular to marketing data, and the particular ones are the ones analysts have to know.
Definition drift. The tool uses a reasonable definition that is not the organisation's: sessions for visits, sign-ups for customers, platform conversions for sales. The number is right by its own lights and wrong for the decision. The metrics layer is the defence.
Time and period errors. Time zones, weeks that start on different days, months compared to partial months, and "last month" interpreted differently from how the asker meant it. These produce confident, tidy, wrong tables.
Correlation read as cause. Narrative tools explain, and marketing data is full of things that move together without one causing the other: spend and seasonality, brand search and TV, email sends and sales. Only a designed test shows cause.
Survivorship and selection. Models trained on the customers who converted describe those customers, not the people who did not, which is a problem for any "what works" analysis drawn from converters only. The survivorship bias framework describes the same trap in research, reporting and benchmarks, and the checks that bring the missing group back.
Tracking treated as truth. A model cannot know that a tag broke, that consent changes shrank the measured population, or that a platform changed its attribution window. It projects the measured series as if it were the real one. The data quality checks come before the analysis, not after.
Double counting across sources. Platform-reported conversions, analytics conversions and CRM records overlap and disagree; a tool asked to combine them will sum them unless told how they relate.
Fluency as evidence. A clear, well-structured answer is more persuasive than a table, and no more likely to be right. The polish of the output is not information about its accuracy.
Verifying AI analysis
Verification is the step that makes any of this usable, and it is cheaper than it sounds when it is routine. Before an AI-produced figure reaches a decision or a report, four things are confirmed. The definition: what exactly was counted, stated in the organisation's terms. The scope: the period, the time zone, the filters and the unit. The source: which table or report, and whether it is the governed one. The reproduction: that the figure can be reproduced from the source by a person following the stated query or steps.
The depth of the check scales with the decision. A figure used for a quick orientation can be checked by reading the query. A figure that moves budget, goes to a client or appears in a board report is reproduced independently. The accuracy and review workflows guide sets out review tiers by risk for all AI output; analysis maps onto them naturally, with reproduced figures at the top tier.
Verification also feeds back. Every mismatch found in review is a lesson about the tool, the data or the prompt, and a short log of them turns into the list of standing instructionsThe business's rules placed before every question an AI assistant answers: scope, sources, refusals, when to hand over and what never to promise. a team gives every query: which definitions to use, which tables are governed, which time zone, which touch model. Over time that log is what makes plain-language querying safe to hand to more people, provided those people can also question a number, which the guide to building data literacy in a marketing team sets out how to teach. The checking AI-written SQL tutorial works through this verification for one generated query, from the definition to reconciliation and a saved, verified version, and the tutorial on checking AI-written formulas and calculated fields does the same for a spreadsheet formula or a dashboard field. When a figure an assistant produced cannot be reproduced at all, the diagnosis for AI-generated report numbers that do not match separates a definition mismatch from a wrong formula, a double-counting query, a missing metrics layer, a skipped check and a small sample.
Common mistakes
- Accepting the table without reading the query. The query is the answer; the table is its consequence.
- Letting the tool define the metric. Conversions, customers and revenue mean specific things in the organisation; the metrics layer, not the tool, holds those meanings.
- Acting on a flag. An anomaly is a reason to look, not a finding.
- Setting alerts at round numbers. Thresholds unrelated to normal variation fire constantly or never.
- Publishing a narrative's causes. Summaries describe; people explain, after a breakdown.
- Treating a forecast as a commitment. Use the range, state the assumptions, and keep score.
- Mixing sources without a rule. Platform, analytics and CRM figures overlap; summing them inflates everything.
- Skipping data quality. A model built on a broken tag projects the break.
The applications section sets analysis beside the other uses of AI in marketing and gives a way to judge each by value, review cost and risk.
