What counts as an AI incident
An AI incidentA live error in marketing work, data or systems that reaches customers, spend or reports, such as a wrong price, a wrong segment or a broken purchase tag., for a marketing team, is any case where output produced or shaped by an AI tool reached customers, or acted on their records, in a way that was wrong, misleading, harmful or outside the rules the team set. The AI governance hub lists an incident list among the records every team should keep; this guide is about what happens before an entry is written.
| Incident type | Typical example | First concern |
|---|---|---|
| Wrong claim or fact | A product page lists a feature the product lacks | Customers buying on a false basis |
| Wrong price or offer | A generated email states a discount that was never approved | Honouring or correcting the offer fairly |
| Wrong personalisation | A message references the wrong purchase, name or circumstance | Who saw what about whom |
| Assistant's wrong answer | A chat assistant states a returns policy that does not exist | Customers acting on the answer |
| Personal data in output | One customer's details appear in another's message or a public post | A possible personal data breach |
| Fabricated social proof | A generated review or testimonial is published | Misleading customers; removal |
| Off-brand or offensive output | An automated reply that is rude, insensitive or bizarre | Reputational spread |
| Automation outside its limits | A workflow sends to the wrong segment or far too often | Volume and recipients affected |
The definition is deliberately wider than "the AI made a mistake". Many incidents involve a person who approved a draft without checking, an input that was wrong before the model saw it, or an automation with no limit. All of them count, because the response and the lessons are the same.
Contain: stop the source
The first move is to stop more people seeing the error. It comes before investigating and before correcting, because every minute the source keeps running adds customers to the correction list.
Containment depends on where the error lives. Take down or revert a page. Pause the email, the automated flow or the scheduled social posts. Switch the chat assistant to a safe fallback, a handover to people or a fixed message, rather than leaving it answering. Pause the ad or ad group. If a feed or template is the source, freeze it. The automation governance guide recommends a documented way to pause every workflow; incidents are when it gets used.
Preserve evidence while you contain. Screenshot the output as customers saw it, export the logs, note the prompt, model and settings version, and record the time the error was found and the time it was stopped. Do not delete the faulty version: you will need it for the trace and possibly for others who have to assess the incident.
Then name an owner for the incident, normally the owner of the use in the AI register, and a single place where everyone working on it records what they find. If the team has no register yet, the AI use register template lays one out with an owner and an incidents column per use.
Correct: fix what customers saw
Correction means fixing the error where people saw it, not only at the source. A corrected product page does not reach the customer who already bought on the old claim, and a fixed assistant does not reach the customer it already misinformed.
Work out who saw what. Use send logs, assistant transcripts, order records and analytics to list the people who received or acted on the wrong output. Then correct in the same channel and with similar prominence: a follow-up email to the recipients of a wrong email, a message to the customers the assistant misinformed, a visible correction note on a page that was widely shared. Say what was wrong and what is right, plainly, without burying it in other news. For a marketing email, the tutorial on fixing a mistake in a sent email sets out when a follow-up is needed at all and how to keep it to the original recipients.
Decide how to treat customers who acted on the error. Where the brand published a wrong price or offer, or an assistant promised something, the fair course and the legal position may both matter, so involve whoever owns commercial and legal decisions. The service recovery and complaints guide covers how to recover a customer relationship after a failure, and its principles apply unchanged: own it, fix it and make it right for the person affected. Where support staff will be offering the remedies, the tutorial on setting remedies and goodwill limits sets what they can give without asking and what goes to an approver. If generated reviews or testimonials were published, remove them; Social ProofEvidence that other people have chosen, used or approved something, used to reduce a buyer's doubt. that was fabricated cannot be corrected, only withdrawn.
Tell: who needs to know
Different people need to know different things, at different times.
Internally, tell the incident owner's manager, the teams customers will contact (service, sales, social), and anyone whose work depends on the same tool or prompt. Give service teams a short note on what happened and what to say, before the correction goes out, so they are not surprised by the replies.
Data protection, immediately, if personal data was involved in any way. The GDPR treats a breach of security leading to accidental or unlawful disclosure of personal data as a personal data breach, and it sets duties on the organisation: to notify the supervisory authority without undue delay and within a short deadline the regulation specifies, unless the breach is unlikely to result in a risk to people; to inform the affected people without undue delay where the risk to them is high; and to document breaches whatever the decision. Those assessments belong to whoever handles data protection, which is why they hear first.
Customers, through the correction itself. Whether to mention that an AI tool was involved depends on whether it helps them understand what happened. An assistant's wrong answer is usually best explained plainly; a drafting error in an email rarely needs it. In either case the brand owns the mistake rather than blaming the tool.
Platforms and partners, if the error appeared in their channels or breached their policies, as their help centres and agreements describe.
Trace: find the cause in the chain
AI output is produced by a chain, and the trace walks it link by link: the input data, the source pack or retrieval, the prompt or template, the model and its settings, the automation logic, the review step and the publication step. The aim is to find where the error entered and why nothing stopped it.
Most incidents have two causes: one that created the error and one that let it through. A model invented a feature (creation) and the product page skipped the fact check because the deadline was tight (escape). A feed carried an old price (creation) and the email automation had no rule comparing prices with the approved offer list (escape). An assistant answered from a stale policy document (creation) and no one had sampled its transcripts in weeks (escape). Fixing only the creation cause leaves the next error free to escape the same way. The academy exercise on handling an AI mistake that reached customers at an opticians chain practises all five moves on one email, tracing each error to the link that created it and the check that let it through.
The NIST AI Risk Management Framework describes this as part of managing AI risk: responding to incidents, learning from them and feeding what is learned back into how systems are governed and measured. The NIST AI Risk Management Framework's manage function is where this response sits beside governing, mapping and measuring each use. The accuracy and review workflows guide catalogues the failure modes that usually sit at the creation end, and the review tiers that should have caught them. For a workflow whose AI step composed or sent the wrong messages, the diagnosis for AI automation that sent the wrong messages walks this trace through trigger, data, prompt, model, checks and send and names the guardrail each cause lacked.
Corriebeck Outdoor's wrong waterproof rating
Corriebeck Outdoor, an invented outdoor clothing retailer, publishes a newsletter with AI-drafted product copy that describes a jacket as fully waterproof; it is water-resistant. A customer emails service 6 hours after the send. The team pauses the scheduled resend and corrects the product page within 1 hour. Send logs show 38,000 recipients; order records show 214 jacket orders since the send. A correction email goes to all recipients the next morning, and the 214 buyers get a personal note offering a free return or exchange; 31 take it. The trace finds the model mixed the jacket with a waterproof model in the same source pack (creation) and the newsletter skipped the claim check because it was treated as a resend of approved copy (escape). The fixes: one product per source pack, and a rule that any edited copy goes through the claim check. Over the following 2 months, churn among the 214 buyers is no higher than among other jacket buyers.
Log and change the system
Close the incident with an entry in the incident list: what happened, when it was found and by whom, how many people were affected, what was contained and corrected, who was told, the creation and escape causes, the changes made and the date they were checked. Keep personal data out of the log itself; refer to records by reference. Errors that did not involve AI deserve the same discipline, and the tutorial on reviewing a marketing error after the fix runs that review for any channel, with one error logOne list across every channel of the errors that reached customers: when found and contained, what customers saw, the checks that missed it, causes and actions. that lets AI incidents sit beside the rest.
Then make sure the changes happen. Each incident should change at least one thing in the system: a check added to a review tier, a rule added to an automation, a source pack cleaned, a prompt fixed in the shared library, a use moved to a higher tier, a tool retired. The AI use policy should list incidents among its review triggers, so that a pattern of incidents changes the policy too.
Track the follow-through, not just the incidents. Time to contain, time to correct, the share of incidents with a completed system change and repeat incidents with the same cause make a better KPIA metric tied directly to an objective. set for governance than the raw count, since a team that finds and reports more incidents may simply be looking harder. Where customers were affected, watch the Churn RateShare of customers at the start of a period who left during it. and complaint volume among them against a comparable group, so you know whether the recovery worked.
Finally, keep the tone of the process blameless. People report incidents early when reporting is treated as useful and late, or never, when it is treated as fault. The person who spots an error is the system working.
Common mistakes
- Investigating before containing. Every minute the source keeps running adds people to the correction list.
- Fixing the source and stopping there. Customers who saw the error still believe it until corrected where they saw it.
- Deleting the evidence. The faulty version, logs and settings are needed for the trace and for anyone else who has to assess the incident.
- Telling data protection late. Where personal data is involved, the GDPR sets duties with short deadlines; they need to hear first.
- Blaming the tool in public. The brand chose to publish the output, and customers expect it to own the mistake.
- Finding the creation cause only. Every incident also has an escape cause, and that is the one that will let the next error through.
- Logging without changing anything. An incident list that does not change the system is a record of repeated mistakes.
- Punishing the reporter. Fear of blame makes people hide errors until customers find them.
