What grounding means
A language model writes by predicting likely text, as how language models generate text explains. On its own it draws on patterns from its training, which rarely include a particular company's current prices, delivery terms or product details. Grounding changes the input: the relevant facts are put in front of the model and it is told to rely on them.
Grounding can be manual or automatic. Manual grounding is what the accuracy and review workflows guide calls a source pack: someone gathers the specification, the claims file and the previous version, pastes them in and instructs the model to use only those. Automatic grounding, usually called retrieval or retrieval-augmented generation, does the gathering by software. It is what sits behind most help-centre assistants, "ask your documents" features and internal knowledge search, and it is the subject of this page.
Why a model needs your documents
There are three reasons a model cannot answer reliably about a business from training alone. Its training text stopped at some point, so recent changes are missing. Company-specific facts were rare or absent in that text, so the model has only a generic impression of what a company like yours might offer. And even facts it saw are stored as a compressed pattern rather than a copy, so details such as numbers, names and conditions are reproduced unreliably.
A team's own documents fill all three gaps. The help centre, product sheets, policy pages, pricing tables and approved claims are First-Party DataData a business collects directly from its own customers, visitors and users. in the broad sense: material the business owns, controls and can keep current. Retrieval lets a model use that material at the moment of answering, so when a document changes, the next answer can change with it, without retraining anything.
Step by step: from question to grounded answer
The detail differs between tools, but most retrieval systems follow the same chain.
- Collect the library. Someone decides which documents the tool may draw on: help articles, product pages, policy documents, internal guides. Anything left out cannot be used; anything left in can be, including documents that should have been retired.
- Split documents into passages. Long documents are cut into smaller sections, often called chunks, so that the tool can supply the part that matters rather than whole files. Where the cuts fall affects what each passage means on its own.
- Index the passages. Each passage is processed so it can be searched. Many tools represent passages by meaning, using the technique explained in how embeddings group similar text; many also use word matching, and some combine the two.
- Find relevant passages for a question. When a question arrives, it is processed the same way and compared against the index. The best-matching passages, usually a handful, are selected. Some tools re-rank them or filter by date, product or permission.
- Assemble the prompt. The selected passages are placed into the model's input alongside the question and instructions such as "answer only from the passages below; if they do not contain the answer, say so; cite the passage for each point".
- Generate and cite. The model writes an answer, and the tool shows citations or links to the passages it supplied.
The table shows what each step contributes and how it can go wrong.
| Step | What it decides | Typical failure |
|---|---|---|
| Collecting | Which material can ever be used | Old or draft documents left in the library |
| Splitting | What each passage contains on its own | A condition or exception cut off from the rule it qualifies |
| Indexing | How passages can be found | Product codes or names matched poorly by meaning-only search |
| Finding | Which passages the model sees | The right passage ranked below a similar-looking wrong one |
| Assembling | What the model is told to do with them | Weak instructions that let general knowledge fill gaps |
| Generating and citing | What the reader is told | Blended passages, or a citation that does not support the sentence |
Where retrieval goes wrong
Retrieval reduces invention; it does not remove it. The failures fall into three groups.
The right material was never found. A customer asks how to "send something back" and the returns article talks about "returns" and "refunds"; a meaning-based index usually bridges that gap, but not always. A question names a product code that a meaning-based search treats as noise. The relevant fact sits in a passage that was split away from its heading, so nothing in it signals what it is about. When nothing relevant is found, a model with weak instructions answers anyway from general patterns, which is exactly the behaviour grounding was meant to prevent.
The wrong material was found. Two versions of a policy are both in the library and the older one matches the question's wording better. A page about one product is retrieved for a question about a similar one. A passage says "this offer ends on Friday" and nothing tells the model which offer or which Friday. These answers are especially convincing because they come with a citation.
The model strayed from the material. It combines two passages into a claim neither makes, adds a plausible detail from its training, or smooths over a condition. Citations do not prevent this, because they usually record which passages were supplied rather than proving which sentence came from which passage.
The support channels and self-service guide warns that bots generating rather than retrieving answers need care; this list explains why even retrieving bots need a review routine and a route to a person. When the logs already show a run of wrong answers, the diagnosis for an AI assistant giving customers wrong answers traces each one to the passage it cited and sorts it into these groups.
A heating installer's assistant quotes an expired guarantee
Kestrelwick Heating, an invented boiler installer, adds a question-answering assistant to its website, grounded in 85 help articles and policy documents. In a review of 120 logged conversations, the team finds 14 answers that state a 7-year guarantee, although the current policy offers 5 years on most models. The library contains both the old and the new guarantee pages, and the old page uses the phrase "how long is my guarantee", which matches customer wording closely. A further 6 answers describe servicing intervals for the wrong boiler range because a single long article covering 3 ranges was split into passages that do not name the range. The team archives the old guarantee page, rewrites the servicing article as 3 short articles, each naming its range in the first line, and adds a test set of 30 real questions that it reruns after every library change.
Keeping the source library fit for retrieval
Because most failures start in the documents, the library deserves the same care as a published site.
- One current source per fact. Duplicate and superseded documents are the commonest cause of wrong answers. Archive old versions out of the library rather than marking them "old" inside it.
- Dates and owners. Every document carries a last-reviewed date and a named owner, so stale material is visible and someone is responsible for it.
- Self-contained sections. Each section names its subject in its heading and first line. Avoid "as above", "this plan" and "see the table" without saying which.
- Tables explained in words. Prices, eligibility rules and comparisons in tables can be split badly or read out of context; a sentence stating the key rule helps.
- Exceptions next to rules. A condition that lives two paragraphs away from the rule it qualifies may not be in the same passage.
- Names and codes spelled consistently. Product names, plan names and codes should match the words customers use, or both should appear.
- A test set of real questions. Questions drawn from search logs and support tickets, rerun after changes, show whether retrieval finds what it should.
The tutorial on preparing a document library for an AI assistant turns these principles into a step-by-step routine for an existing pile of documents, from the first inventory to tying updates to price and policy changes. The academy checkpoint on grounding and standing instructions tests reading a wrong answer back to the library, from superseded versions and tables without sentences to sections that cannot stand alone.
What it means for you
- Treat the document library as part of the product. The quality of a grounded assistant is mostly the quality, freshness and structure of what it searches.
- Ask what a tool retrieves from and how it handles no match. "Say you do not know" is a design choice; check that it has been made.
- Open citations when the answer matters. A citation shows what the model was given, not that the sentence is supported.
- Measure with real questions. The share of test questions answered correctly from current sources is a sensible KPIA metric tied directly to an objective. for a retrieval tool, tracked over time.
- Choose retrieval for facts that change. Prices, stock, policies and offers belong in maintained documents; the retrieval vs fine-tuning comparison covers the alternative.
Common misconceptions
- "A grounded tool cannot make things up." It makes things up less, but it can still blend, stray or answer from general patterns when retrieval finds nothing.
- "A citation proves the answer." It shows which passages were supplied; the sentence still needs checking against them when it matters.
- "More documents mean better answers." More outdated or overlapping documents mean more chances to retrieve the wrong one.
- "Retrieval means the AI was trained on our content." The model is unchanged; it reads passages at question time.
- "If the answer is wrong, the model needs replacing." Most wrong answers trace to missing, stale or badly split documents, which are cheaper to fix.
