First, trace a sample of wrong answers to the passage each one cited
Pull a sample of conversations: every one flagged by a customer or an agent, plus a handful chosen at random from the same weeks. For each wrong answer, record the question as the customer typed it, the answer, the passage or article the assistant cited, and whether the library holds a passage that would have answered correctly. Then open the cited passage and read it. How grounding and retrieval work sets out why that one step does most of the diagnosis: a retrieval tool splits the library into passages, finds the ones that best match the question, places them in front of the model with an instruction to answer from them, and shows a citation to the passages it supplied. The citation records what was supplied, not which sentence came from which passage, so reading the passage is the only way to see whether the answer followed it.
Four outcomes cover almost every wrong answer. The passage says what the answer says and is out of date: the library holds a superseded version, or the current article is itself wrong. The passage is about something else, a different plan or product, or a section cut off from its heading: retrieval found the wrong material. Nothing relevant was cited, or the answer holds facts no passage holds: the right material was never found and the model answered anyway, which points at the instructions. The passage is right and the answer is still wrong: the model strayed, and the instructions and the test set are where to look. Across the sample, count how many fall into each group. A run of wrong answers usually has one dominant group, and the cause behind it is the one to fix first.
Before touching anything, name who owns what. A RACI matrix with a row per decision does it in minutes: the owner of each policy, price or product is accountable for the document that describes it; the assistant's owner is responsible for the library as a whole, the instructions, the test set and the sampling; the service lead is consulted on wording and hand-over topics and informed when a sample finds something; whoever administers the tool is responsible for its settings and connections. The NIST AI Risk Management Framework, as the site's framework page applies it, calls the trace and the matrix mapping and governing: knowing where the assistant acts, what went wrong and who answers for it, before changing anything. What to say to the customers who already received a wrong answer is the subject of the guide to handling an AI mistake that reached customers; this page is about why the answers keep being wrong.
Diagnosis table
Diagnosis
| Symptom | Check | Likely cause | Fix |
|---|---|---|---|
| The answer matches a policy or price that was true last year, and the cited passage is a superseded version | The library for other versions of the same document, and which version the citation points at | Old and new versions of a policy both in the library, so the superseded passage is retrieved | Pick one current source for each fact and archive every other version outside the library |
| The cited passage is about a different product or plan, or nothing was cited and the assistant answered anyway | Whether a right passage exists, names its subject in its heading and first line, and uses the customer's words | The right passage never found or the wrong one found, and the model answering anyway | Rewrite sections to stand alone, put rules and tables into sentences, use customers' words, and instruct the assistant to say when the library is silent |
| Answers draw on general knowledge, describe another company's policy or contain facts the library does not hold | The standing instructions for a sources rule, a silence rule and a hand-over list | Standing instructions that never told the assistant to answer only from the library and to say when it cannot | Write the sources rule, the silence rule and the hand-over list into versioned instructions with a test conversation for each |
| Wrong answers began after a library change, a new plan or a model update, and nobody noticed until a customer did | Whether a test set of real questions exists, when it was last run and against what | No test set of real questions run before launch or after a library change | Build a test set from chat logs and tickets, score it against a pass bar, and rerun it after every change |
| Customers report the errors, repeat contacts rise, and transcripts show nobody asked for or reached a person | The hand-over triggers, the route to a person and the weekly sampling record | No handover to a person and no weekly sampling of answers, so errors are found by customers | Hand over when the library is silent or the customer asks, and read a weekly sample against the cited content |
| The cited passage is the current article, and the article itself is wrong | The article against the live policy, price or product, and its owner and review date | A help centre that was already out of date before the assistant read it | Run the help centre as a product with an owner and a review date per article, tied to the release and price-change process |
Fix, verify and prevent each cause
Two versions of a document in the library
The tutorial on preparing a document library for an AI assistant names duplicate and superseded documents as the commonest cause of wrong answers, and the grounding page explains the mechanism: retrieval matches a question to the passages that best fit its wording, and the old version of a policy often fits a customer's question better than the new one. An instruction to prefer recent material does not fix it, since that depends on the old document saying it is old and on the model following the instruction every time, while retrieval may still place the old passage in front of it.
Fix: group the documents by the fact they describe, returns, delivery, guarantees, prices, each product, and where a group holds more than one version, decide with the policy's owner which document is the source of truthThe system whose figure is treated as authoritative for a given number, such as billing for revenue or the CRM for pipeline, with other tools reconciled to it. for that fact. Archive every other version outside the library, not marked "superseded" inside it, and merge two documents that each hold half of an answer into one. Note in the register what was archived and where it went.
Verify: rerun the questions from the sample that cited a superseded passage; each answer should now cite the current document, and the sentence should match what that document says.
Prevent: keep a register with one row per document, its owner, its last-reviewed date and the fact it is the source for, so a new version replaces the old one in the library rather than joining it, and show the review date on the document itself where the system allows, so data freshnessWhether a table, feed or report holds data as recent as it should; a freshness check fails when the latest date is older than the source's expected arrival. is visible to anyone who reads it.
The right passage never found, or the wrong one found, and the model answering anyway
How grounding and retrieval work sorts retrieval failures into three groups, and the sample will show which one the assistant has. The right material was never found: the customer asks how to "send something back" and the article says "returns"; a question names a product code that a meaning-based search treats as noise; the fact sits in a passage split away from its heading, so nothing in it signals what it is about. The wrong material was found: a page about one product is retrieved for a question about a similar one, or a passage says an offer ends on Friday and nothing tells the model which offer or which Friday. The model strayed: it combined two passages into a claim neither makes, or smoothed over a condition. When nothing relevant is found, a model with weak instructions answers anyway from general patterns, which is exactly the behaviour groundingMaking a language model answer from specified material, such as documents retrieved for the question, rather than from what it absorbed in training. was meant to prevent.
Fix: work on the documents first, as the preparing tutorial does. Give each section a heading that names the product or plan and the question it answers, with the answer in the first lines; replace "as above" and "this plan" with the actual names; split a long document that covers several products into one per product; put prices, eligibility rules and comparisons that live in tables into sentences, with each exception in the same paragraph as the rule it qualifies; and use the words customers use beside the internal names, spelled the same way in every document.
Verify: put the questions from the sample to the assistant again and trace each answer to its passage; the passage should be the one that holds the fact, with its subject named in its heading.
Prevent: keep a set of real questions drawn from chat logs, tickets and site search, rerun it after every change to the library, and track the share of those questions answered correctly from current sources.
Instructions that never limited the assistant to the library
The tutorial on writing the standing instructions for an AI assistant explains what makes an assistant's instructions different from a prompt in a team's library: they are followed in every conversation with no one reading the reply before the customer does, so they have to carry the decisions a colleague would otherwise make. The core rule, which the customer service guide gives in a sentence, is that the assistant answers only from the supplied documents, names the article or page it used, says plainly when the documents do not cover a question, and then offers a person. Instructions without that rule leave the model free to fill a gap from its general training, which may describe another company's policy.
Fix: write the sources rule and the silence rule into the instructions, say what to do when two sources disagree, which is to stop and hand over rather than choose, and write the hand-over list as topics and signals rather than a request for good judgement. Keep facts out of the instructions, because a price or a policy written there goes stale where no document owner will see it. Add the list of things the assistant never does: promising a refund, a credit, an exception or a delivery date beyond what a cited document states, asking for details the business does not need, or presenting itself as a person.
Verify: write a test conversation for each rule and one that tempts the assistant to break it, including a question the documents nearly answer and a question they do not answer at all, and run them in the test environment; the silence case should end in a plain statement that the documents do not cover it and an offer of a person.
Prevent: give the instructions a version number, a date and an owner, keep every earlier version, record what changed and why, and rerun the test conversations whenever the instructions, the documents or the underlying model change. When a test fails, change the instructions or the documents, never the test.
No test set run before launch or after a library change
The guide to testing prompts and models makes the point that a demo shows what a tool can do on a good day and a test shows how often it does the job on yours. An assistant launched on the strength of a few good answers, or changed without a rerun, is noticed failing only when customers notice. The guide's method is a test setA collection of real inputs for an AI task, each noting what a good output must contain, run to compare prompts or models and rerun when they change. drawn from real work, a rubric written before anything is run, a comparison that removes bias, a failure log and a pass bar agreed in advance.
Fix: build the set from the assistant's own logs and the support team's tickets: the typical questions, the hard ones with exceptions and specialist terms, the edge cases where the assistant should decline because the documents are silent, and the sensitive cases where a mistake matters most, such as prices, eligibility and legal wording. For each, write the facts that must appear, the facts that must not, and the passage the answer should come from. Score with criteria that are pass or fail for factual accuracy against the source and for handling of gaps, so that one invented guarantee is never averaged away by good tone.
Verify: run the set, read the failure log before the scores, and check that the failures match the wrong answers in the sample; if they do not, the set is missing the cases customers actually raise and needs them added.
Prevent: store the set beside the instructions and rerun it when the documents are restructured, a price or policy changes, the instructions are edited, the vendor changes the model or the team switches tools, or reviewers notice the errors changing shape. The AI customer service launch checklist asks for this test, and a plan to rerun it, before the assistant answers its first customer, and an assistant that has been fixed is relaunched through the same gate.
No handover and no weekly sampling
The AI in customer service guide treats the handover as the place an assistant is judged. Hand over automatically when the content does not cover the question, when the customer asks for a person, when the conversation loops, when sentiment turns sharply negative and when any topic kept for people appears; the person who takes over receives the full conversation, so the customer does not repeat themselves; and when no person is available, the assistant says so honestly and offers a callback or an email with a stated response time. Disclosure sits beside it: say at the start, and whenever asked, that the customer is talking to an automated assistant. The European Commission's overview of its regulatory approach includes transparency expectations so that people know when they are interacting with an AI system, and the FTC's business guidance holds that deception standards do not change with the technology; the site states both only at the level of direction, and none of this is legal advice. Disclosure does not make answers right, which is why the guide pairs it with a weekly sample read by a person: a random sample of automated conversations plus every conversation that ended in a handover, a complaint or a negative rating, each checked for whether the answer matched the cited content, whether the content was correct and current, whether the handover happened when it should and whether the tone fitted.
Fix: add the hand-over triggers and the route to a person before anything else, because they stop the next wrong answer about money or eligibility reaching a customer unchallenged; then book the sample as a weekly routine with a named reader and a route for findings: content gaps to the document owner, routing errors to whoever maintains the rules, recurring failures to the decision about scope.
Verify: a test question outside the library ends in a handover with the transcript carried across, and the first weekly sample is read, logged and acted on.
Prevent: measure what customers experience rather than how many contacts the assistant closed: issues resolved without a repeat contact, the share of sampled answers that matched current content, and how quickly a customer who wanted a person reached one, read by contact reason. Where the question is whether an assistant should answer first at all, the AI chat assistant test designs a randomised comparison with sampled accuracy, time to reach a person and repeat contacts as its guardrails.
A help centre that was already out of date
Sometimes the trace ends at the current article, faithfully cited, and the article is wrong. The support channels and self-service guide says why: maintenance is where most help centres fail, articles go out of date when the product, prices or policies change, and an out-of-date article is worse than none because it creates contacts and damages trust. A bot drawing on the help centre is only as good as the help centre, and once an assistant reads it, the same maintenance decides what customers are told.
Fix: correct the article with the person who owns the policy, price or product it describes, and run the help centre as the guide describes, as a product with users, a purpose, measures of success and a backlog: the questions customers ask most often, in their words, written first; one question per article with the answer at the top; an owner and a review date on each.
Verify: the corrected article matches the live policy, and the assistant's answer to the sample question now matches the article.
Prevent: link help-centre maintenance to the release and price-change process, so a change to the product triggers a change to the help before the assistant starts giving the new answer, and read the assistant's unanswered questions as the help centre's backlog, since the questions it cannot answer show the gaps. The launch checklist checks the help content behind an assistant before it goes live, and the tutorial on preparing a document library is the clean-up the help centre needs before an assistant reads it.
Prevention: a library with one owner per document and a test set rerun before every price or policy change
Every cause above is a document nobody owned, an instruction nobody wrote, a test nobody ran or a sample nobody read, and the routine that prevents the next run of wrong answers gives each of those a person and a date.
The library comes first. The register from the preparing tutorial, one row per document with its owner, its last-reviewed date and the fact it is the source of truth for, is the record that stops a second version creeping back in. The owner is the person who changes the policy, product or price, because that is the person who knows when the article is wrong, not whoever set up the tool. The RACI row for each document says so, and the assistant's owner holds the library as a whole. Grounding in the sense the glossary entry gives it, making the model answer from specified material, is only as good as that material, which is why the register matters more than any setting in the tool.
The change procedure comes second. Every process that changes a fact customers ask about, a price change, a new product, a policy revision, a seasonal offer, gets one more line: the owning document is updated, its review date is reset and the test questions that touch it are rerun before the change goes live. A change log of what changed in the library and why lets a later wrong answer be traced to the day it started. The instructions are versioned in the same way, with an owner and a rerun of their test conversations whenever the instructions, the documents or the model change.
The sample comes third. A person reads a fixed sample of conversations every week, including every conversation that ended in a handover, a complaint or a negative rating, and logs what they find by cause: a document, a retrieval gap, an instruction, a handover that did not happen. Findings go to the owner who can fix them. The measures that go to the people who decide the assistant's scope are resolution, accuracy against current content and handover quality by contact reason, never deflection, because an assistant that is hard to get past scores well on deflection while service gets worse. The human reviewThe step where a named person checks AI-assisted work for accuracy, sources, originality, privacy and disclosure before it is published or sent. of a conversational tool is this sampling plan, since its answers cannot be reviewed one by one before they are sent.
Around the routine sits the launch gate and the wider section. The AI customer service launch checklist gathers scope, content, handover, disclosure, data and testing into one list to tick before an assistant first answers customers and again after a fix; the foundations hub explains how the tools behind it work. The AI troubleshooting hub lists the neighbouring symptoms. Where the wrong facts are in drafts rather than conversations, the diagnosis for AI content that contains invented facts traces them to the sources, the prompt and the review; where an automated workflow has sent the wrong message to the wrong people, the diagnosis for AI automation that sent the wrong messages starts from the send log.
A self-storage chain finds the old terms its assistant preferred (illustrative)
Zephyrlake Storage
Zephyrlake Storage, an invented self-storage chain with 7 sites, notices in one month that 5 customers at check-in quote an insurance rule its website assistant gave them: that a home contents policy covers stored goods automatically. The rule changed 14 months earlier; stored goods now need the chain's own cover or a named extension to the customer's policy.
The service lead pulls 40 conversations from the month, 9 flagged by staff and 31 at random, and traces each answer to its cited passage. 23 are right. 11 cite a passage from one of 2 superseded versions of the storage terms still sitting in the connected folder; the oldest version's heading, "Is my stuff insured while it's in storage?", is closer to how customers ask than the current page's "Insurance requirements". 4 answer a question the library never covered, about access on bank holidays, from nowhere in particular, and 2 give a different site's opening hours. None of the 40 ended in a handover, and the instructions hold no line telling the assistant to decline when the library is silent.
The chain archives the 2 old versions outside the library and records the current terms page as the source of truth for insurance, owned by the operations manager. The instructions are rewritten as version 2, with a sources rule, a silence rule and a hand-over list that includes anything about insurance claims, and a test conversation for each rule. The service lead builds a test set of 30 questions from the chat logs; the first run fails 6, all about the 3 sites whose hours lived only in a table, which is rewritten in sentences, and the second run passes all 30. A route to a person is added with the transcript carried over, and the service lead reads 25 conversations every Monday. The price and policy change procedure gains a line: update the terms page, reset its date and rerun the 8 insurance questions before any change goes live. The following quarter's samples find 2 wrong answers in the 300 read, both traced to a new site's page that had no owner yet.
Common mistakes
- Fixing the instructions when the document is wrong. If the trace leads to a stale or unclear passage, fix the passage.
- Marking old versions as superseded instead of removing them. Retrieval matches wording, and the old version often matches the customer's wording best.
- Trusting the citation. It shows which passage was supplied, not that the sentence is supported by it.
- Putting prices and policies in the instructions. Facts belong in owned documents the assistant cites.
- Hiding the route to a person. A customer who has to defeat the bot first turns a wrong answer into a complaint.
- Testing once at launch. The documents, the instructions and the model all change; each change needs a rerun.
- Reporting deflection as success. Customers who give up look like savings until they leave.
