Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › AI Governance for Marketing Teams

Fact-Checking an AI Answer Step by Step

A routine for checking an AI tool's answer before you use it: split it into claims, find primary sources, check quotes, figures and dates, mark the unknowns.
Tutorialbeginner8 min readUpdated

When a single answer needs checking

Not every answer needs a formal check. An answer used to generate ideas, suggest an outline or point you towards sources is a starting point, and reading it critically is enough. The routine below is for answers whose content is about to travel: a competitor's pricing that will go into a positioning deck, a market statistic for a business case, a regulation summary for a brief, a quote for a thought-leadership draft, a list of platform features for a recommendation.

Those answers need checking because of how language models produce text. They generate the most plausible continuation of the question, which is often right and always fluent, and fluency carries no signal about accuracy. The NIST AI Risk Management Framework treats the validity and reliability of AI output as something to be measured rather than assumed, and for an individual marketer that comes down to a simple habit: treat every factual statement in an answer as unverified until a source you have opened supports it.

A useful test is to imagine the answer's claims in your own name, in front of the person most likely to know they are wrong. If that thought is uncomfortable, check the answer.

Before you start

  • The original question and answer. Copied in full, not summarised. The wording of the question shapes the answer and often explains its errors.
  • Access to primary sources. The places facts actually come from: regulators' and official sites, company filings and pages, platform help centres, the original study, your own analytics and CRM.
  • A document to work in. A simple table works: claim, type, source, status, note.
  • Time. A short answer with a handful of claims takes a few minutes; a long research answer can take much longer. If the time is not available, use fewer of the answer's claims.
  • The accuracy guide read. The failure modes in the accuracy and review workflows guide tell you what to look for.

The check, step by step

  1. Copy the answer and the question

    Paste the question and the full answer into your working document, with the tool, the date and any files or sources you gave it. If the tool searched the web, keep the list of sources it showed. The date matters because answers about prices, features, rules and people go out of date, and the tool's knowledge has a cut-off you may not know. Keeping the question also lets you see whether it led the answer, for example by assuming something that was not true.

  2. Split it into claims

    Go through the answer sentence by sentence and pull out every statement that could be true or false, one per row. A single sentence often holds several: "The market grew strongly last year, driven by younger buyers, according to a major industry survey" contains a growth claim, a cause claim and a source claim. Leave out opinion and framing, which are yours to judge rather than verify. Splitting is the most useful step in the routine, because errors hide inside sentences that are mostly right.

  3. Mark each claim's type

    Label each claim: a figure, a date, a quote, a name or attribution, a description of what a product or rule does, a cause, or a source citation. The type decides the check. Figures need the original number, its unit and its date; quotes need the exact words; causes need evidence of the relationship, not just of the two facts; citations need the source opened. Mark which claims you actually intend to use. Claims you will not use can be dropped rather than checked.

  4. Find the primary source

    For each claim you will use, find where the fact originates, not where it is repeated. A statistic comes from the study or dataset that produced it, not a blog post that quotes it; a platform feature from the platform's own documentation; a rule from the regulator's guidance; a company's price from its own page. If the answer gave a citation, open it and read the cited place, then follow it back if it is itself quoting something else. Expertise and trust in content, the qualities summarised as E-E-A-T, rest on this: a claim traced to its origin. When the answer is only the start of a wider look at a market, the desk research tutorial carries the same habit into a source log for every publication the work relies on.

  5. Check quotes word for word

    Find every quotation in its original source and compare it word for word. Models paraphrase, merge two statements into one, attribute a line to the wrong person and occasionally produce a quotation that does not exist anywhere. A quote that is close in meaning but not exact is still a misquote. If you cannot find it, it is not a quote; either paraphrase the source you did find, attributed fairly, or leave it out.

  6. Check figures, dates and units

    Compare each figure with the source: the number itself, its unit, what it counts, the population, the period and the date. Common errors are a percentage reported as percentage points, a regional figure presented as global, an old figure presented as current, a forecast presented as a result and a survey result from a small Sample Size presented as a market fact. Check that dates are right and that anything time-sensitive, such as prices, features and rules, is still current at the source today.

  7. Mark what stays unknown

    Some claims will not trace to any source you can find. Mark them unknown rather than wrong or right. An unknown claim does not go into the slide, brief or decision as fact. It can be removed, rewritten as a question ("we have not confirmed whether…"), or passed to someone who can verify it, such as the analyst who owns the data or the colleague who knows the market. Also note claims that were wrong, and what the source actually says, since a corrected fact is often more useful than the original.

  8. Record the check

    Save the working table with the answer: each claim, its status (verified, corrected, unknown or dropped), the source with a link or location, and the date checked. Note in the slide or brief that its figures were checked against primary sources on that date. The record lets a colleague rely on your check instead of repeating it, and lets anyone trace a figure if it is questioned later. If your team tracks review quality, the share of claims that failed the check is a useful KPI for which tools and question types to trust least.

Red flags in AI answers

Some features of an answer raise the odds that a claim is wrong. None proves an error; each is a reason to check that claim first.

Red flagWhat often lies behind itCheck
A precise figure with no sourceA plausible number assembled from patterns, not a studyFind the original study or drop the figure
A citation that is hard to findA source that does not exist, or a real title with invented detailsSearch for it directly; open it if found
A real source cited for a specific claimThe source exists but says something different or nothing on the pointRead the cited place, not just the title
A quotation with a famous nameA misattribution or a line nobody saidFind it in an original source
"Recent" or "currently" without a dateInformation from before the tool's knowledge cut-offCheck the source as it is today
A neat cause and effectTwo facts joined by an assumed linkLook for evidence of the relationship itself
A list of features or rulesSome items invented or out of dateCheck each against official documentation
An answer that agrees with the question's assumptionThe question led the answerAsk the question neutrally and compare
Confident tone on a contested subjectUncertainty smoothed awayLook for the range of views in sources
Illustrative example — not a real result

Quinbury Garden Supply checks a competitor summary

A marketing manager at Quinbury Garden Supply, an invented online garden retailer, asks an assistant to summarise three competitors' delivery offers and loyalty schemes for a positioning deck. The answer runs to 300 words. Split into claims, it holds 17 statements, of which the manager will use 11. Checking each against the competitors' own delivery and loyalty pages takes 25 minutes. 6 claims are verified, 3 are corrected (one competitor's free delivery threshold had changed, a loyalty scheme gave points per order rather than per pound, and a "next-day" offer applied only to one region), 1 is unknown (a claim that a competitor "recently cut prices by 10%", with no source anywhere) and 1 is wrong (a loyalty scheme that had closed). The deck uses the 9 verified or corrected claims, notes the check date, and leaves the price-cut claim out.

Common mistakes

  • Checking the answer as a whole. "It looks right" is not a check; split it into claims first.
  • Accepting a link as proof. Open the source and read the cited place.
  • Stopping at a secondary source. A blog that quotes a figure is not the figure's origin.
  • Letting near-quotes through. Close in meaning is still a misquote.
  • Ignoring units and dates. Points and percentages, regional and global, old and current are the commonest figure errors.
  • Asking the tool if it is right. The same process that made the error produces the reassurance.
  • Using unknowns as facts. An unverified claim belongs in the deck as a question or not at all.
  • Keeping no record. Without the working table, the next person repeats the check or, worse, trusts the slide.

Frequently asked questions

If the AI tool gives a link to its source, is the answer checked?

No. A link is a pointer, not a verification. Open it and read the cited place: the page may not exist, may be about something else, or may say something different from what the answer attributes to it. Answers that combine a real source with a claim the source does not make are among the hardest errors to spot, precisely because the link looks like evidence.

Do I need to check every AI answer this carefully?

No. Check in proportion to where the answer is going. An answer used to brainstorm, or to suggest where to look, needs little more than reading with care. An answer whose facts, figures or quotes will appear in a slide, a brief, a published piece or a decision needs this routine, because once the claim leaves the chat window it gains authority it has not earned.

Can I ask the AI tool to check its own answer?

You can ask it to list its claims or to suggest where each could be verified, which speeds up the first steps of this routine. Asking it whether its answer is correct is not a check, because the same process that produced the error produces the reassurance. Verification has to come from a source outside the tool that you have opened and read yourself.

Terms used on this page

Where this page is used

Also related

Sources

  1. AI Risk Management Framework — NIST

Related