Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › Troubleshooting

AI Automation Sent the Wrong Messages

Why an AI step in a workflow sent off-brand or wrong messages: offers left to the model, an agent allowed to act, no rate limit, no sampling and stale sources.
Troubleshootingintermediate17 min readUpdated

First, trace one wrong message back through trigger, data, prompt, model, checks and send

This page starts where the guide to handling an AI mistake that reached customers reaches its fourth move. That guide's first three, contain the source, correct what customers saw and tell the people who need to know, are assumed done or under way, and are not repeated here. The trace is the move that decides whether the incident repeats, and the incident guide gives it a shape, treating the output as the end of a chain and walking it one link at a time, from the input data and the source material through the prompt or template, the model and its settings, the automation logic, the review step and the send. In most incidents one cause creates the error and another lets it through, and repairing only the first leaves a gap the next error can pass through.

Pick one wrong message and reconstruct its run. What triggered it, and was the trigger the event the workflow was built for? Which record fields reached the prompt, and were they right at the moment of sending? Which source material did the model see, and was it the current version? What did the prompt allow the model to write, and what did it allow it to change? Which automated checks ran on the output, and which were missing? What permitted the send: a rule, an agent's own decision or a person? Open the logs the automation guide asks every acting step to keep, the input the model saw and the reason it gave, and read the run in that order. If the logs do not exist, that is the first finding.

Then carry the message back with the five whys. The email offered a discount because the offer line was composed by the model; the model composed it because the template left the offer line open; the line was open because the brief treated the offer as copy rather than as a rule; nobody noticed because no sample of sent messages was being read; nothing bounded the damage because the send ran without a volume alarm. The fix sits at the last why for each branch, and the guardrail that should have stood there is what the diagnosis table names. The NIST AI Risk Management Framework, as the automation guide applies it, encourages the habit this trace depends on: treat every model as a component with known failure modes rather than as a clever colleague, map where it is used, measure how it behaves and manage what the measure shows. The applications section sets automation beside the other uses of AI and the review cost each carries; this page assumes the message has already gone out.

If the batch reached people who had opted out, keep that branch separate. The tutorial on fixing a mistake in a sent email treats mail sent to people who had opted out as a consent matter before it is anything else, and the ICO's direct marketing guidance, as the site's guides state it, explains the UK rules on electronic marketing, including offering a way to opt out in every message. That branch ends at the suppression rail, not at the prompt, and belongs with whoever handles data protection.

Diagnosis table

Diagnosis

SymptomCheckLikely causeFix
The message carries a discount, price, claim, legal line or footer that was never approvedWhich parts of the template were fixed and which the model was allowed to writeOffers, prices, claims or legal copy left to the model instead of locked as rulesLock those blocks as rules and confine the model to approved variants or a tight brief
Messages went out, or records changed, with no person or rule in betweenWhat permitted the send or the change: a rule, an approval step or the agent's own decisionAn agent allowed to send or change records where a fixed workflow with an approval step was neededMove sending and record changes to rules with an approval step; grant the agent permissions per action
A whole segment received the bad batch inside an hour, and nobody was alertedThe send volume for the hour against the usual rate, and whether any limit or alarm existedNo rate limit, dry run or volume alarm, so a bad hour reached a whole segmentAdd a rate limit that pauses for review, a dry-run mode and a volume alarm to the owner
The tone or wording has been drifting for weeks and the complaint is the first anyone heardWhen a person last read a sample of sent messages, and what the sample foundNo weekly sample of sent messages read by a person, so drift went unnoticedSet a sampling plan with a cadence, a reader, a failure log and a pause threshold
The message states a promotion, policy or product fact that was true onceWhich document the model composed from, its version and its ownerThe model composing from stale or wrong source materialKeep one current source per fact, archive the rest and tie updates to price and policy changes
The problem began on the day the prompt, model or vendor changedThe change log for the workflow and the date the regression set was last runA prompt, model or vendor change made without rerunning the regression setRerun the stored test set against the recorded baseline before any change goes live

Fix, verify and prevent each cause

The AI for marketing automation guide draws the line plainly: offers, prices, legal copy, claims about the product and the unsubscribe footer are fixed, and the model chooses between approved variants, or writes within a tight brief, only for the lines where tone and relevance matter and an error is survivable. A template that leaves the offer line open invites the model to compose one, and a model composes fluently whether or not the offer exists. The same applies to a merge field: the guide to personalising email content gives every field a fallback written for its sentence, and a field the model is allowed to rewrite is no longer a field.

Fix: rebuild the template so that offers, prices, claims, legal lines, footers and merge fields are rules the model cannot touch, and give the model a short list of approved opening lines or a brief with the facts it may use and nothing else.

Verify: generate the message for a seed contact with the model instructed to be as creative as it can be, and confirm the locked blocks come out identical every time while only the permitted lines vary.

Prevent: run every generated element through the automated checks the automation guide lists before send, banned terms, claim patterns, length, links that resolve and placeholders left unfilled, and treat any template that opens a locked block to the model as a reviewable change.

An agent allowed to send or change records where a fixed workflow was needed

The comparison of rule-based automation and AI agents answers the question most incidents of this kind turn on: anything that sends, spends or changes records belongs to rules, and an agent is used only where the steps cannot be written down in advance and a person approves the result before it reaches a customer. An agent that was given the whole task, qualify, write, send and update, has been given permissions the comparison would refuse, and its mistakes are varied rather than systematic, which is why the same run can send a good message to one customer and a wrong one to the next.

Fix: separate drafting from acting, as the automation guide puts it: the agent may read, classify and draft, and a rule or a person sends, changes the stage, applies the offer or deletes anything, with permissions granted per action rather than per agent.

Verify: run the workflow with sending switched off and read what the agent proposes for a varied set of records, several runs each, before any action is switched back on; then confirm every action it keeps is logged with the input it saw and the reason it gave.

Prevent: list every agent in the automation inventory with its permissions, as the governance guide asks, name an owner who reads its logs and can pause it, and treat each new task it is given as a new deployment with its own test and its own permissions.

No rate limit, dry run or volume alarm

A workflow that can send to everyone it matches will do so in the hour the fault appears. The automation governance guide names the volume alarm as the place most quiet failures become visible first: a workflow that sends far more or far fewer messages than usual in a day should notify its owner, and a scheduled report comparing the day's counts with the recent average does the job where the platform offers nothing. The automation guide adds the two controls that turn a bad hour into a bad minute for an acting step: a rate limit, so a workflow can take only a bounded number of actions before pausing for review, and a dry-run mode that shows what it would have done.

Fix: add a rate limit that pauses the workflow for review once a bounded number of sends has gone out, a dry-run mode the owner can switch on after any change, and a volume alarm to the workflow's owner.

Verify: raise the trigger for more seed contacts than the limit allows and confirm the workflow stops and asks, and that the alarm reaches a person who will act on it the same day.

Prevent: make the rate limit, the dry run and the alarm part of the inventory row for every workflow with an AI step, and check them in the governance guide's weekly read of volume alarms; the frequency cap and the global suppression check every send should pass stay as rails on every workflow.

No weekly sample of sent messages read by a person

A model that composes at send time produces messages nobody on the team has seen, and the automation guide's answer is a sample of sent messages read by a person each week, because that is the only way to catch a model whose tone has drifted. The accuracy and review workflows guide sets the customer-facing and automated tier's review as exactly this: guardrails in the system, a sampled review on a schedule, an incident route and an owner who reads the logs, with the sampling plan deciding the sample size, the cadence and who reads it, logging what the sample found by failure mode, and setting a threshold at which the use pauses for a full review. Drift that ran for weeks is the signature of a sample nobody read.

Fix: write the sampling plan down, name the reader, start the failure log, and set the pause threshold before the workflow is switched back on.

Verify: read the first week's sample against the brand's voice guidelines and the locked facts, and confirm the log records each defect by failure mode with the prompt and model version beside it.

Prevent: keep the human review at that tier permanent rather than a launch-week habit, and read the failure log monthly for the rise in defects that signals a change nobody announced.

The model composing from stale or wrong source material

A message can be wrong because the model did its job on the wrong input. The accuracy guide calls these inherited errors, an old spec or a wrong field reproduced with confidence, and the tutorial on preparing documents for an AI assistant shows how they start: a library holding old and new versions of the same promotion, a document with no owner and no review date, a policy that changed while the article did not, and a superseded passage that matches the template's wording better than the current one. A model asked to compose from that library will quote whichever version it is given.

Fix: list every document the workflow's model can reach, pick one current source per fact with its owner, archive every other version outside the library rather than marking it superseded, and give each remaining document an owner and a last-reviewed date.

Verify: generate the message for a seed contact and trace every fact in it to the current source; a citation shows what the model was given, not that the sentence is supported, so open the passage.

Prevent: tie document updates to the processes that change prices, promotions and policies, so the owning document is updated and the workflow's test questions rerun before the change goes live, and keep a change log of the library so a later wrong message can be traced to the day it started.

A prompt, model or vendor change made without rerunning the regression set

The automation guide calls the regression test set, a fixed collection of inputs with known good outputs rerun whenever the prompt, model or vendor changes, the cheapest control on its list and the one most teams skip. The guide to testing prompts and models explains why it is needed: vendors change the models behind their tools, sometimes quietly, so a prompt that once passed can start failing. The guide's trigger list covers an edited prompt or shared context block, a model change announced or chosen, changed tool settings, restructured source documents, and reviewers noticing a new kind of error. A workflow whose wrong messages began on the day of such a change has told you its cause.

Fix: rerun the stored test set against the recorded baseline, apply the pass bar agreed in advance, including the failures that are unacceptable at any rate such as an invented price or offer, and roll back or repair the prompt until it passes.

Verify: compare the new failure log with the baseline's and confirm the failure type that produced the incident no longer appears; then switch the workflow back on in dry-run mode for its first runs.

Prevent: put the rerun inside change control, so no prompt, model, vendor or settings change reaches a live workflow without a logged rerun and an owner's sign-off, and store the test set beside the prompt so a rerun is a routine task rather than a project.

Prevention: locked blocks, permission per action, a rate limit and a regression set before any change

Every cause above is a guardrail the automation guide describes and the workflow lacked, and the prevention routine is to put all of them in front of every workflow with an AI step rather than only the one that failed. The incident guide's last move, log it and change the system, and the governance guide's last incident step, check whether the same cause exists elsewhere in the inventory, are the same instruction: walk the inventory and ask of each workflow which of the six guardrails it carries. The AI automation go-live checklist puts those guardrails into one gate to tick before the paused workflow is switched back on and before any new AI step goes live.

Start with the template. Offers, prices, claims, legal lines, footers and merge fields are rules; the model chooses between approved variants or writes within a brief that lists the facts it may use. Every generated element passes the automated checks before send. A template that opens a locked block to the model is a reviewable change with a second person's sign-off, in the same way the governance guide treats a change to the suppression list or the frequency cap.

Then the permissions. Sending, spending and record changes are rules with an approval step; an agent drafts, classifies and proposes, and holds permissions per action, logged with the input it saw and the reason it gave. Each agent has an inventory row listing its permissions, an owner who reads its logs weekly and can pause it, and a rule that each new task is a new deployment. The comparison's four questions, whether the task sends, spends or changes records, whether the steps can be written down, whether a person can check the result before it matters, and whether the team can test, monitor and own it, are asked again whenever the task grows.

Then the limits. A rate limit that pauses for review, a dry-run mode for the first runs after any change, a volume alarm to the owner, the frequency cap and the global suppression check before every send. The alarm, the cap and the suppression check are the governance guide's rails, applied to every workflow rather than left to each designer; the rate limit and the dry run are the automation guide's controls for any step that acts. Read the volume alarms weekly in minutes, and treat an unusual hour as something to open the same day.

Then the evidence. A sampling plan per automated use, with its reader, cadence, failure log and pause threshold, and a stored test set per prompt, rerun under change control whenever the prompt, model, vendor, settings or source documents change, with the result logged and signed off. Pick a KPI at each of the three levels the automation guide measures: the share of outputs approved unedited or passing the sample at the component level, incidents and their severity at the operational level, and the flow's own outcome against a holdout at the workflow level, because a flow reporting no incidents and almost no review time is usually one nobody watches. Keep the documents the model composes from under the same discipline, with one current source per fact, an owner and a review date, and a line in every price, promotion and policy change that updates the document and reruns the tests before the change goes live. When the same gaps show up in an assistant answering customers, the diagnosis for an AI chatbot giving wrong answers traces its own chain. When the model's output is ad creative a platform has turned down rather than a message a customer has received, the diagnosis for rejected AI-generated ad creative applies. The AI troubleshooting section lists the neighbouring symptoms. The tutorial on writing a sampling plan for an automated AI use walks through a sampling plan for one use, from naming the reader to setting the pause rule.

A holiday park operator finds the two gates its retention flow lacked (illustrative)

Illustrative example — not a real result

Molquessa Holiday Parks

Molquessa Holiday Parks, an invented operator of 12 holiday parks, runs a retention workflow that sends a check-in email to past guests whose usual booking window is approaching. The model composes the opening line and a suggested-stay paragraph at send time inside a template. One evening the workflow sends 4,180 emails in a little over an hour. Guests reply within minutes: 1,930 of the emails offer 20% off the next stay, which the parks never agreed, and 212 address the guest by another guest's first name.

The team pauses the workflow, sends a correction the next morning and honours the discount for anyone who books within the week. Then it traces one message. The offer line was composed by the model from a promotions document in the shared folder that still described a spring campaign, alongside the current one; the greeting came from a merge field the template allowed the model to rewrite, and in 212 runs the model wrote a name from the suggested-stay paragraph's example instead; and the send ran at full speed with no rate limit and no volume alarm, 3 days after the vendor updated the model behind the tool, a change nobody had tested.

The fixes follow the trace. The offer block and the footer become fixed rules; the greeting becomes a merge field with a fallback the model cannot touch; the model is restricted to choosing one of 6 approved opening lines and composing the suggested-stay paragraph from a brief that names the parks and nothing else. The spring promotions document is archived outside the folder and the current one gets an owner and a review date. A rate limit pauses the workflow for review after 300 sends in an hour, a volume alarm goes to the workflow's owner, and 25 sent messages are read every Monday with each defect logged by type. The team stores a regression set of 40 guest records with known good outputs and reruns it after any prompt or model change; the first rerun, after the next vendor update 7 weeks later, catches 2 outputs in which the model tried to add a discount to the suggested stay, before the workflow is switched back on.

Common mistakes

  • Fixing the prompt and switching the workflow back on. The prompt created the error; the missing guardrail let it through, and it is still missing.
  • Leaving the offer line, the legal line or a merge field open to the model. Anything the model can rewrite, it will.
  • Giving an agent the whole task. Sending and record changes are rules with an approval step; the agent drafts.
  • Running a send-time model at full speed with no limit. A bounded number of actions before a pause turns a bad hour into a bad minute.
  • Treating the launch-week sample as the review. The sample is permanent, with a reader, a log and a pause threshold.
  • Trusting a document because it is in the folder. One current source per fact, with an owner and a date; the rest archived outside.
  • Letting a vendor's model update go live untested. The stored test set, rerun under change control, is how the change is noticed before customers notice it.
  • Tracing only the workflow that failed. The same guardrail is usually missing elsewhere in the inventory.

Frequently asked questions

We have paused the workflow and corrected the message. Why trace it at all?

Because containing and correcting deal with the message that went out, and the trace deals with the next one. Most incidents of this kind have a cause that created the error and a cause that let it through, and fixing only the first leaves the second in place for a different error to find. The trace also tells you whether the same gap exists in other workflows, which is the step most teams skip and the one that stops the repeat. A paused workflow with no trace is usually switched back on with the same guardrail still missing.

Should the model be removed from the workflow after a mistake like this?

Remove the model from the step where it had freedom it should never have had, not from the workflow as a whole. Offers, prices, claims, legal lines and footers become fixed blocks; sending and record changes become rules with an approval step; the model keeps the bounded job it was good at, such as choosing between approved opening lines or classifying a reply. If the trace shows the model cannot be made reliable for its bounded job either, the step goes back to a person or a rule until a test set says otherwise.

Some of the messages went to people who had opted out. Is that the same problem?

It is the same workflow and a different kind of incident. A message to someone who opted out is a consent matter before it is anything else, whatever the wording of the message, and it needs the suppression check that every send should pass rather than a better prompt. Involve whoever handles data protection, keep the send logs, and treat the gap in the suppression rail as its own cause with its own fix. The wording causes on this page still apply to everyone else who received the batch.

Terms used on this page

Frameworks

Where this page is used

Sources

  1. AI Risk Management Framework — NIST
  2. Direct marketing guidance — Information Commissioner's Office (UK)

Related