Why the documents decide most of an assistant's answers
GroundingMaking a language model answer from specified material, such as documents retrieved for the question, rather than from what it absorbed in training. through retrieval works in a chain that the grounding and retrieval page sets out: the documents in the library are split into passages and indexed, the passages most relevant to a question are fetched, often by similarity of meaning as how embeddings group similar text explains through the EmbeddingA list of numbers that represents a piece of text so that texts with similar meaning sit close together, used for search, retrieval and clustering. technique, and they are placed in front of the model with an instruction to answer from them. Every link in that chain depends on what the library holds. A passage that was never written cannot be found; a superseded passage that matches a customer's wording can be found first; a condition separated from its rule can be left behind.
Grounding narrows invented answers without removing them, and the same page records that most failures trace to missing, stale or badly split documents rather than to the model. That is good news for a marketing or service team, because documents are something it already knows how to fix. When an assistant is already giving customers wrong answers, the diagnosis for an AI assistant giving customers wrong answers traces a sample of conversations to the library, retrieval, the instructions, the test set and the handover before this clean-up begins. The work is ordinary content maintenance, done with a reader in mind who takes every sentence literally and never asks a colleague what was meant.
The table links the common faults in a library to what the assistant does with them and the step below that deals with each.
| Fault in the library | What the assistant does with it | Step that fixes it |
|---|---|---|
| Old and new versions of the same policy | Quotes whichever passage matches the wording best | Pick one current source per fact |
| Customer details or internal notes in a document | Repeats them to whoever asks | Take out personal data and internal-only material |
| No owner and no review date | Keeps answering from a page nobody has checked | Give every document an owner and a date |
| Sections that say "as above" or "this plan" | Answers from a passage that never names its subject | Rewrite sections so each one stands alone |
| Prices or eligibility only in a table | Misreads or mismatches the cells | Put rules, exceptions and tables into sentences |
| Internal product names customers never use | Misses the passage the question was about | Use the words customers use |
| A policy changed but the article did not | Gives the old answer confidently | Tie document updates to policy and price changes |
None of this replaces the other parts of a working assistant. The standing instructionsThe business's rules placed before every question an AI assistant answers: scope, sources, refusals, when to hand over and what never to promise. it follows, such as answering only from the library and saying when it does not know, are the subject of the tutorial on writing instructions for an AI assistant; the method for scoring its answers is the guide to testing prompts and models; and the scope, handover and disclosure rules are in the AI in customer service guide. This page covers the documents.
Before you start
- The list of sources the assistant reads. Whoever set up the tool can say which folders, sites or collections it is connected to; ask for the actual list rather than the intended one.
- Access to the documents and their history, including the help centre's admin view, the shared drives and any internal wiki the assistant reads.
- The current policies, prices and product facts from the people who own them, so you can tell which version is the live one.
- A set of real questions drawn from support tickets, chat logs and site search, as the guide to testing prompts and models describes building a test set.
- The team's data rules, including the tutorial on minimising data before prompting, because a document library is material entering an AI tool like any other.
- A sheet for the library register this routine builds, one row per document. If the team already keeps a content inventory for its published pages, such as the content inventory template, extend it rather than starting a second list.
The steps
List every document the assistant can reach
Start from the connections, not from what the team thinks is in there. Export or write down every document in every source the assistant reads: help articles, policy pages, product sheets, uploaded files, internal guides and anything added during a trial and forgotten. Record for each its title, location, type and last-edited date. This list is the real boundary of what the assistant can say. The NIST AI Risk Management Framework, as the site's framework page applies it, calls this mapping: knowing where AI is used and what could go wrong before deciding what to do about it. The steps that follow are its manage function: acting on what the map shows.
Take out personal data and anything it must not quote
Read the list for documents that hold customer names, order details, complaint notes, staff contact details or internal commentary such as margin targets, supplier terms and draft plans. Anything the assistant can reach it can repeat, so a document that should never be read to a customer does not belong in the library at all, however useful it is to staff. Remove personal data at source or move the document out of the connected folders, following the data minimisation tutorial. Where an internal document holds one useful fact, write that fact into a customer-facing document and leave the internal one out.
Pick one current source for each fact and archive the rest
Group the documents by the fact or policy they describe: returns, delivery, guarantees, prices, each product. Where a group holds more than one version, decide with the policy's owner which document is the Source of TruthThe system whose figure is treated as authoritative for a given number, such as billing for revenue or the CRM for pipeline, with other tools reconciled to it. for that fact and archive every other version outside the library. Do not leave old versions in place marked "superseded"; the grounding page records that a duplicate matching the customer's wording can be retrieved ahead of the current page. Where two documents each hold half the answer, merge them into one. Note in the register what was archived and where it went.
Give every document an owner and a last-reviewed date
For each document left in the library, name the person who owns the policy, product or price it describes, and record the date it was last checked against the live facts. A RACI matrix helps where the work is shared: the document owner is accountable for its content, the assistant's owner is responsible for the library and its tests, and service leads are consulted on wording. Show the review date on the document itself where the system allows, so Data FreshnessWhether a table, feed or report holds data as recent as it should; a freshness check fails when the latest date is older than the source's expected arrival. is visible to anyone reading it. A document without an owner is the one that goes stale first, which the support channels and self-service guide also warns about for help centres in general.
Rewrite sections so each one stands alone
Retrieval hands the model passages, not whole pages, so a passage has to make sense without the page around it. Give each section a heading that names the product or plan and the question it answers, and put the answer in the first lines. Replace "as above", "this plan", "the previous option" and "see below" with the actual names. Split long documents that cover several products into one document per product, each naming its product at the top. The help centre and documentation SEO guide gives the same advice for articles customers find in search, so the rewrite helps human readers too.
Put rules, exceptions and table contents into sentences
Tables, footnotes and conditions far from their rule are where meaning is lost when a document is split. For each table of prices, sizes, eligibility or comparisons, add a sentence or two that states the key rule in words. Move each exception into the same paragraph as the rule it qualifies, so a passage that carries the rule also carries the exception. Where a rule depends on a date, a region or a plan, say so in the sentence rather than in a heading above it. Keep the table for people who scan; the sentences are for the passage that travels alone.
Use the words customers use for products and plans
Compare the questions in your test set with the names used in the documents. Customers ask about "the small unit" or "sending something back" while the documents say a product code or "returns authorisation". Meaning-based search often bridges that gap but not always, and product codes in particular can be matched poorly. Where the customer's word differs from the internal one, use both in the heading or first line, and spell product and plan names the same way in every document.
Run real questions and trace each answer to its passage
Put the test questions to the assistant and, for each answer, open the passage it cites or the document it should have used. Record whether the answer is right, whether it came from the current source, and if not, which document it came from. Wrong answers usually trace to something the earlier steps should have caught: an archived version still connected, a table without a sentence, a section without a name. Fix the document, then run the questions again. The testing guide covers scoring and pass bars; here the point is the trace, because a citation shows what the model was given, not that the sentence is supported.
Tie document updates to policy and price changes
A library cleaned once goes stale as soon as a price or policy moves. Add a line to every process that changes a fact customers ask about, such as a price change, a new product, a policy revision or a seasonal offer: the owning document is updated, its review date reset and the test questions that touch it rerun before the change goes live. Keep a Change LogA dated record of changes made to tracking, accounts, reports, lists or rules, with who made each and why, so later movements in the figures can be traced. of what changed in the library and why, so a later wrong answer can be traced to the day it started. The guide to updating and correcting content sets the house rules for how published changes are made and shown, and the AI customer service launch checklist has the go-live checks this routine feeds.
The routine is the document half of an assistant. Whether to answer from a maintained library at all, rather than a model trained on the business's material, is the question in retrieval vs fine-tuning; for facts that change, the library is the answer, and this is how it is kept. The foundations hub places it among the other pages on how AI tools work. The academy exercise on cleaning the library behind a bicycle retailer's help assistant practises the routine end to end, from listing what the assistant can reach to tracing failed test answers to the passages that caused them.
A self-storage company cleans its assistant's library (illustrative)
Ibrovash Self Storage traces an old insurance rule and cleans its library
Ibrovash Self Storage, an invented self-storage business with 4 sites, adds an assistant to its website that answers from its help centre and a shared folder of policy documents. Within its first month, the service lead notices customers quoting an insurance rule that changed the previous spring.
Listing what the assistant can reach finds 62 documents, including 3 versions of the storage terms, 2 of them superseded, and a folder of 9 internal pricing notes that someone connected during the trial. The trace shows the assistant quoting the oldest terms, whose wording matched the customer question "do I need my own insurance" most closely.
| Change | Before | After |
|---|---|---|
| Storage terms | 3 versions in the library | 1 current version; 2 archived outside it |
| Internal pricing notes | 9 notes connected | Removed; the 2 public facts they held moved into the prices article |
| Owners and review dates | None recorded | Each of the 51 remaining documents has an owner and a date |
| Unit-size table | Table only, with internal codes | Each size described in a sentence with its everyday name |
| Test questions | None | 25 real questions from chat logs, each traced to its passage |
On the first run, 19 of the 25 answers trace to the current passage. The other 6 trace to 3 causes: a table without sentences, 2 sections that said "as above" and a unit size customers call "the garage one" that appears nowhere in the documents. After those fixes, all 25 trace correctly. The operations manager adds a line to the price-change procedure: update the prices article, reset its date and rerun the 7 questions that mention prices before new prices go live.
Common mistakes
- Cleaning the documents you meant to connect, not the ones that are connected. Start from the actual list of sources; trial uploads and forgotten folders are where the surprises are.
- Marking old versions instead of removing them. A superseded document in the library can still be retrieved; archive it outside.
- Leaving internal notes in because staff find them useful. Whatever the assistant can reach it can quote to a customer.
- Owners in name only. The owner is the person who changes the policy, not whoever set up the tool.
- Trusting the citation. It shows which passage was supplied; open it and check that it says what the answer claims.
- Fixing the instructions when the document is wrong. If the trace leads to a stale or unclear passage, fix the passage.
- Treating the clean-up as a project with an end. Without a line in the price and policy change processes, the library drifts back within a season.
