Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › Advanced

Grounding a Help Assistant for a Bicycle Retailer

You decide where an invented bicycle retailer's product question assistant should get its facts and its voice, turn a vendor's promise of a model trained on the brand into questions, clean the library the assistant answers from and trace its wrong answers back to the documents that caused them. You finish with a mechanism decision, a vendor question list, a library register with owners and dates, rewritten passages and a line in the price-change procedure that keeps the library current.
ExerciseAdvanced levelModule 1: Grounding assistants in the business's own knowledge8 minUpdated

Why this matters

The intermediate AI in marketing level chose tools on evidence and designed pilots with a decision rule. This exercise starts after the choice, with the question that decides whether an assistant is right in the details customers act on: where does the business's knowledge live? The comparison of retrieval and fine-tuning answers it. Retrieval leaves the model unchanged and places the relevant passages from the brand's documents in front of it for each question, so facts stay current and answers can cite their source. Fine-tuning trains a model further on the brand's examples, which shapes style and format but carries facts only as an impression. The tutorial on preparing a document library then makes the point that decides most of the work: an assistant that answers from documents knows nothing the documents do not say, and most of its wrong answers start in the library it reads from.

The retailer and its two proposals

Nivquade Cycles is an invented online retailer of bicycles, parts and accessories. It has a help centre, product pages with specifications, a few policy pages and a shared folder that someone connected during a trial. Two vendors have proposed a product question assistant. The first retrieves from the help centre and product pages and cites the page it used. The second offers "a model trained on your brand" built from past product copy and customer emails, and its demo sounds more like the retailer than the first vendor's does. The owner likes the voice. The service lead has run both on a test set of real questions from chat logs, with names removed.

Illustrative example — not a real result

Nivquade Cycles: what the assistant can reach, and what the test runs returned

Source connected to the assistantDocumentsWhat the list shows
Help centre articles483 versions of the returns policy, 2 of the delivery policy
Product pages310Frame sizes in a table with internal codes; 22 pages for discontinued models still live
Policy pages7Guarantee terms with the exceptions in a footnote
Shared folder "assistant trial"14A margin-by-brand sheet, an order export with customer names, a supplier price list and 2 draft blog posts

Test set: 40 real questions.

ResultRetrieval vendor"Trained on your brand" vendor
Right, with a source the lead could open290 (no source to open)
Right in substance2925
Said it did not know and offered a person50
Wrong on a size, a price, compatibility or a policy615, of which 9 concerned discontinued models
Sounded like the retailer, in the lead's judgement2136

The second vendor confirms that removing a discontinued model from its answers means another round of training.

Facts in documents, form in instructions

The comparison's decision rule is a sort. List what the assistant must know and separate facts from form. Sizes, compatibility, prices, delivery, returns, the guarantee and the approved claims are facts: they change, customers act on them and a reviewer may ask where an answer came from, so they belong in retrieval, where Grounding keeps each answer tied to a passage the retailer owns and can update. Voice and format are form: handle them with prompts, approved examples and the standing instructions first, and consider Fine-Tuning only for a measured, stable gap in form that remains after those have been tried, with enough approved examples to train on and the capacity to re-tune and re-test when the base model changes. Most tuned set-ups still use retrieval for facts.

The library then decides the quality of the retrieval. The tutorial's routine is the order of work: list what is connected, take out what must not be quoted, keep one current Source of Truth per fact, name an owner and a review date for each document, rewrite sections to stand alone, put tables into sentences, use the customers' words, trace test answers to passages and tie updates to the changes that move the facts.

Task

  1. Sort what the assistant must know into facts and form, choose the mechanism for each, and write the questions the retailer must put to the second vendor before "trained on your brand" can mean anything in a contract.
  2. Start from the connected sources, not the intended ones. Decide what leaves the library and why.
  3. Group the remaining documents by the fact they describe, pick one current source per fact with the person who owns it, archive the rest outside the library and record an owner and a last-reviewed date for every document that stays.
  4. Rewrite the passages the test run shows cannot travel alone: the size table, the guarantee footnote and the product customers call by a name that appears in no document.
  5. Trace each of the retrieval vendor's wrong answers to its cause, fix the document rather than the instructions, and say what the re-run should show.
  6. Add the line to the price and policy change process that keeps the library current, and write the change-log entry for this clean-up.

Worked answer

Facts and form. Everything in the retailer's questions that can be wrong is a fact: a frame size, whether a part fits a model, a price, a delivery time, a returns condition. Those go to retrieval over owned documents. The voice the owner liked is form, and the comparison says to handle it with a voice guide, approved examples and instructions before anything is trained. The second vendor's accuracy on discontinued models is the comparison's warning in practice: a tuned model holds an impression of its examples, and removing information from it means re-tuning. The decision is retrieval, with a few approved voice examples in the instructions and a re-test later.

The vendor questions. The comparison lists them: which mechanism sits behind the promise and for which features; where the documents and examples are stored, for how long and who at the vendor can see them; whether the retailer's material trains models offered to anyone else, and whether that can be switched off; how a changed price or policy reaches answers and how quickly; how a document or example is removed; what happens when the base model changes; whether the vendor will run the retailer's own test set and report the results; and what the retailer can take away at the end. The vendor terms on inputs and training belong in the agreement, as the disclosure and data privacy guide sets out.

What the assistant can reach. The trial folder leaves the library whole: the margin sheet and supplier prices are internal-only, the order export is personal data that must be removed at source following the data minimisation tutorial, and the draft posts are not approved facts. Where a trial document held a public fact, that fact is written into a customer-facing article and the document stays out. The discontinued product pages are archived outside the library too: a page that matches a customer's wording can be retrieved ahead of the current one, so "superseded" labels are not enough.

One source per fact, with an owner. The returns and delivery policies each keep one version, chosen with the person who sets the policy; the other versions are archived outside the library and the register records where they went. Each remaining article and product page gets the owner of the fact it describes and the date it was last checked against the live facts, shown on the document where the system allows, so Data Freshness is visible to anyone reading it.

Passages that stand alone. The size table gains sentences that state the rule for each size in words, with the everyday size name beside the internal code. The guarantee's footnoted exceptions move into the same paragraph as the rule they qualify. The folding model customers ask about by a nickname gets that name in the heading and first line of its page, spelled the same way everywhere.

Illustrative example — not a real result

Nivquade Cycles: tracing the retrieval vendor's 6 wrong answers

Cause found by opening the cited passageAnswersFix
The oldest returns policy matched the question's wording21 current version; 2 archived outside the library
Frame size read from the table alone2A sentence per size with its everyday name
Guarantee exception sat in a footnote1Exception moved into the rule's paragraph
"The folding one" appears in no document1Nickname added to the page heading and first line

Re-run: 38 of the 40 answers trace to the current passage. The remaining 2 are questions no document covers, where the assistant says it does not know and offers a person. The team writes the 2 missing articles and adds the questions to the set.

The trace. Each wrong answer is read back to the passage the assistant was given, because a citation shows what the model was handed, not that the answer is supported. Every cause is a library fault the earlier steps describe, so the fix is to the document, never to the instructions. Scoring and pass bars for the re-run follow the guide to testing prompts and models.

Keeping it current. A line goes into every process that changes a fact customers ask about: update the owning document, reset its review date and rerun the test questions that touch it before the change goes live. The change-log entry records what was archived, what was rewritten and why, so a later wrong answer can be traced to the day it started. The instructions the assistant follows, with their silence rule, hand-over list and disclosure line, are a separate job in the tutorial on writing standing instructions, and the AI customer service launch checklist ticks off both before customers see the assistant. The mechanism this whole exercise rests on is in how grounding and retrieval work.

Common mistakes

  • Choosing the vendor whose demo sounds right. Voice is form; the test set showed where the facts were wrong.
  • Fine-tuning to teach facts. Discontinued models, prices and sizes drift into plausible errors that re-tuning alone can remove.
  • Cleaning the documents you meant to connect. The trial folder was the surprise, and it held personal data.
  • Marking old versions instead of removing them. The oldest returns policy was still retrieved first.
  • Fixing the instructions when the document is wrong. Every traced cause was a passage, not a rule.
  • Treating the clean-up as finished. Without the line in the price-change process, the library drifts back within a season.

The checkpoint that follows in this module tests the same judgements.

Pages this lesson applies

Terms used on this page

Also related

Related