Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › How AI Works for Marketers

How Grounding and Retrieval Work

How AI tools answer from your own documents: indexing, retrieving relevant passages, adding them to the prompt, citing them, and where the process still fails.
How it worksintermediate8 min readUpdated

What grounding means

A language model writes by predicting likely text, as how language models generate text explains. On its own it draws on patterns from its training, which rarely include a particular company's current prices, delivery terms or product details. Grounding changes the input: the relevant facts are put in front of the model and it is told to rely on them.

Grounding can be manual or automatic. Manual grounding is what the accuracy and review workflows guide calls a source pack: someone gathers the specification, the claims file and the previous version, pastes them in and instructs the model to use only those. Automatic grounding, usually called retrieval or retrieval-augmented generation, does the gathering by software. It is what sits behind most help-centre assistants, "ask your documents" features and internal knowledge search, and it is the subject of this page.

Why a model needs your documents

There are three reasons a model cannot answer reliably about a business from training alone. Its training text stopped at some point, so recent changes are missing. Company-specific facts were rare or absent in that text, so the model has only a generic impression of what a company like yours might offer. And even facts it saw are stored as a compressed pattern rather than a copy, so details such as numbers, names and conditions are reproduced unreliably.

A team's own documents fill all three gaps. The help centre, product sheets, policy pages, pricing tables and approved claims are First-Party Data in the broad sense: material the business owns, controls and can keep current. Retrieval lets a model use that material at the moment of answering, so when a document changes, the next answer can change with it, without retraining anything.

Step by step: from question to grounded answer

The detail differs between tools, but most retrieval systems follow the same chain.

  1. Collect the library. Someone decides which documents the tool may draw on: help articles, product pages, policy documents, internal guides. Anything left out cannot be used; anything left in can be, including documents that should have been retired.
  2. Split documents into passages. Long documents are cut into smaller sections, often called chunks, so that the tool can supply the part that matters rather than whole files. Where the cuts fall affects what each passage means on its own.
  3. Index the passages. Each passage is processed so it can be searched. Many tools represent passages by meaning, using the technique explained in how embeddings group similar text; many also use word matching, and some combine the two.
  4. Find relevant passages for a question. When a question arrives, it is processed the same way and compared against the index. The best-matching passages, usually a handful, are selected. Some tools re-rank them or filter by date, product or permission.
  5. Assemble the prompt. The selected passages are placed into the model's input alongside the question and instructions such as "answer only from the passages below; if they do not contain the answer, say so; cite the passage for each point".
  6. Generate and cite. The model writes an answer, and the tool shows citations or links to the passages it supplied.

The table shows what each step contributes and how it can go wrong.

StepWhat it decidesTypical failure
CollectingWhich material can ever be usedOld or draft documents left in the library
SplittingWhat each passage contains on its ownA condition or exception cut off from the rule it qualifies
IndexingHow passages can be foundProduct codes or names matched poorly by meaning-only search
FindingWhich passages the model seesThe right passage ranked below a similar-looking wrong one
AssemblingWhat the model is told to do with themWeak instructions that let general knowledge fill gaps
Generating and citingWhat the reader is toldBlended passages, or a citation that does not support the sentence

Where retrieval goes wrong

Retrieval reduces invention; it does not remove it. The failures fall into three groups.

The right material was never found. A customer asks how to "send something back" and the returns article talks about "returns" and "refunds"; a meaning-based index usually bridges that gap, but not always. A question names a product code that a meaning-based search treats as noise. The relevant fact sits in a passage that was split away from its heading, so nothing in it signals what it is about. When nothing relevant is found, a model with weak instructions answers anyway from general patterns, which is exactly the behaviour grounding was meant to prevent.

The wrong material was found. Two versions of a policy are both in the library and the older one matches the question's wording better. A page about one product is retrieved for a question about a similar one. A passage says "this offer ends on Friday" and nothing tells the model which offer or which Friday. These answers are especially convincing because they come with a citation.

The model strayed from the material. It combines two passages into a claim neither makes, adds a plausible detail from its training, or smooths over a condition. Citations do not prevent this, because they usually record which passages were supplied rather than proving which sentence came from which passage.

The support channels and self-service guide warns that bots generating rather than retrieving answers need care; this list explains why even retrieving bots need a review routine and a route to a person. When the logs already show a run of wrong answers, the diagnosis for an AI assistant giving customers wrong answers traces each one to the passage it cited and sorts it into these groups.

Illustrative example — not a real result

A heating installer's assistant quotes an expired guarantee

Kestrelwick Heating, an invented boiler installer, adds a question-answering assistant to its website, grounded in 85 help articles and policy documents. In a review of 120 logged conversations, the team finds 14 answers that state a 7-year guarantee, although the current policy offers 5 years on most models. The library contains both the old and the new guarantee pages, and the old page uses the phrase "how long is my guarantee", which matches customer wording closely. A further 6 answers describe servicing intervals for the wrong boiler range because a single long article covering 3 ranges was split into passages that do not name the range. The team archives the old guarantee page, rewrites the servicing article as 3 short articles, each naming its range in the first line, and adds a test set of 30 real questions that it reruns after every library change.

Keeping the source library fit for retrieval

Because most failures start in the documents, the library deserves the same care as a published site.

  • One current source per fact. Duplicate and superseded documents are the commonest cause of wrong answers. Archive old versions out of the library rather than marking them "old" inside it.
  • Dates and owners. Every document carries a last-reviewed date and a named owner, so stale material is visible and someone is responsible for it.
  • Self-contained sections. Each section names its subject in its heading and first line. Avoid "as above", "this plan" and "see the table" without saying which.
  • Tables explained in words. Prices, eligibility rules and comparisons in tables can be split badly or read out of context; a sentence stating the key rule helps.
  • Exceptions next to rules. A condition that lives two paragraphs away from the rule it qualifies may not be in the same passage.
  • Names and codes spelled consistently. Product names, plan names and codes should match the words customers use, or both should appear.
  • A test set of real questions. Questions drawn from search logs and support tickets, rerun after changes, show whether retrieval finds what it should.

The tutorial on preparing a document library for an AI assistant turns these principles into a step-by-step routine for an existing pile of documents, from the first inventory to tying updates to price and policy changes. The academy checkpoint on grounding and standing instructions tests reading a wrong answer back to the library, from superseded versions and tables without sentences to sections that cannot stand alone.

What it means for you

  • Treat the document library as part of the product. The quality of a grounded assistant is mostly the quality, freshness and structure of what it searches.
  • Ask what a tool retrieves from and how it handles no match. "Say you do not know" is a design choice; check that it has been made.
  • Open citations when the answer matters. A citation shows what the model was given, not that the sentence is supported.
  • Measure with real questions. The share of test questions answered correctly from current sources is a sensible KPI for a retrieval tool, tracked over time.
  • Choose retrieval for facts that change. Prices, stock, policies and offers belong in maintained documents; the retrieval vs fine-tuning comparison covers the alternative.

Common misconceptions

  • "A grounded tool cannot make things up." It makes things up less, but it can still blend, stray or answer from general patterns when retrieval finds nothing.
  • "A citation proves the answer." It shows which passages were supplied; the sentence still needs checking against them when it matters.
  • "More documents mean better answers." More outdated or overlapping documents mean more chances to retrieve the wrong one.
  • "Retrieval means the AI was trained on our content." The model is unchanged; it reads passages at question time.
  • "If the answer is wrong, the model needs replacing." Most wrong answers trace to missing, stale or badly split documents, which are cheaper to fix.

Frequently asked questions

Is retrieval the same as training the AI on our documents?

No. Retrieval leaves the model unchanged and shows it relevant passages from your documents each time a question is asked, so updating a document changes the next answer. Training or fine-tuning changes the model itself, and the model then carries an impression of the material rather than the material. When a tool says it uses your content, it is worth asking which of the two it means, because they behave very differently when facts change.

Why does a grounded assistant still cite the wrong document?

Because the citation records which passages were supplied, not which passage each sentence actually came from. The model may blend two passages, draw a detail from its general training, or attach a citation to the nearest plausible source. Some tools do this better than others, but none removes the need to open the cited passage and check that it says what the answer claims when the answer matters.

How should we write help articles so an AI assistant uses them well?

Write sections that make sense on their own: a heading that names the product and the question, the answer in the first lines, and no reliance on "as above" or a table that only makes sense with the page around it. Keep one current article per topic, date it, give it an owner and archive the old version rather than leaving both. Those habits help human readers as well, which is why they are worth doing anyway.

Terms used on this page

Where this page is used

Also related

Related