What a language model is and is not
A language model is a system that has learned, from examples, which pieces of text tend to follow which. Given some text, it can estimate what is likely to come next, and by doing that repeatedly it can write paragraphs, answer questions, summarise documents and rewrite copy. The chat tools marketers use wrap a model in an interface, add instructions the user never sees, and sometimes add other components such as search or file reading. The model itself is the part that turns text in into text out.
It is useful to be clear about what that description excludes. A language model is not a database: it does not store documents and retrieve them, and it has no list of verified facts it consults before answering. It is not a search engine, although some tools put a search step in front of it. It does not understand a brief the way a colleague does, with knowledge of the business, the customer and last week's meeting. And it does not know when it is wrong, because it has no separate process for checking its output against the world.
None of that makes it less useful. A system that has absorbed the patterns of a vast amount of writing is very good at structure, phrasing, tone, summarising what it is given and producing many variations quickly. The AI for content guide shows where those strengths fit marketing work. The point is to use it for what the mechanism delivers.
Training: learning patterns from text
Training is the process that gives a model its abilities, and in general terms it has two stages. In the first, the model is shown an enormous amount of text and repeatedly asked to predict missing or next pieces of it. Each time it predicts badly, its internal settings are adjusted slightly so that it would have predicted better. Over a very large number of adjustments, it comes to encode a great deal about language: grammar, style, how arguments are structured, which words go with which topics, and a lot of facts that appeared often and consistently in the text.
In the second stage, most models used in commercial tools are trained further on examples of instructions and good responses, and on people's judgements about which of several answers is better. This is what turns a general text predictor into an assistant that follows requests, adopts a helpful tone and declines some tasks. It shapes behaviour; it does not add a fact-checking layer.
Two consequences follow. First, what the model "knows" is a compressed impression of its training text, not a copy of it. Facts that appeared many times in consistent form are usually reproduced well; facts that were rare, recent, disputed or specific to one company are reproduced poorly or not at all. Second, the training text stops at some point. Anything that happened afterwards, including a product launch, a price change or a new platform policy, is outside what the model learned unless a tool supplies it in the prompt.
Step by step: from prompt to answer
When a marketer types a request, roughly the following happens. The detail varies between tools, but the principle is shared.
- The tool assembles the input. The visible prompt is combined with instructions set by the tool's maker, any earlier messages in the conversation and any attached files or retrieved passages. All of it becomes one long piece of text.
- The text is split into tokens. Tokens are small units, often a word or part of a word, plus punctuation. The model works on tokens rather than letters or whole sentences.
- The model scores every possible next token. Using what it learned in training, it estimates how likely each token in its vocabulary is to come next, given all the input so far.
- One token is chosen. The tool picks a token from the likely candidates. Usually it does not always take the single most likely one; a controlled amount of randomness keeps the writing varied. Settings in some tools turn that randomness up or down.
- The chosen token is added to the text, and the process repeats. The model now scores the next token given the input plus everything it has written so far. It continues until it produces a token that means "stop" or reaches a length limit.
The table summarises what each stage means for someone reading the output.
| Stage | What happens | What it means for the output |
|---|---|---|
| Input assembly | Instructions, conversation and material are joined into one text | Anything not included is invisible to the model |
| Tokenising | Text is split into small units | Exact spellings, codes and numbers can be handled awkwardly |
| Scoring | Each possible next token gets a likelihood | Common phrasing and common claims are favoured |
| Choosing | One token is picked, with some randomness | Two runs of the same prompt can differ |
| Repeating | Each token is chosen given everything before it | An early wrong turn is built on, not corrected |
The last row deserves emphasis. The model does not plan a whole answer and then write it out; it commits to each piece as it goes. If an early sentence states something false, the following sentences tend to support it, because consistent continuations are likely continuations.
Why fluent text can be false
Put the steps together and the cause of confident errors, often called hallucination, is visible. The model is always producing the most plausible-looking continuation. When the true answer is in front of it or was well represented in training, the plausible continuation is usually the true one. When it is not, the plausible continuation is something that has the shape of a true answer: a statistic with a believable value, a quote in the right voice, a study with a convincing title, a product feature that a product of that kind might have.
Nothing in the output marks the difference. A recalled fact and a composed one are produced by the same process, in the same fluent style, with the same confident tone. That is why proofreading catches almost none of these errors and why the accuracy and review workflows guide treats every figure, name and claim in AI output as unverified until a source is shown. When invented facts keep turning up in a team's drafts, the diagnosis for AI content containing invented facts separates missing sources, a thin prompt, fluency-only review, stale retrieval and an untested change.
Some situations make invention more likely, and they are common in marketing work:
- Requests for specifics the model cannot have. A company's own prices, policies, results or customer quotes.
- Requests for sources. Citations follow a predictable pattern, so an invented one is easy to produce.
- Niche or recent topics. Sparse training text means a weak pattern, filled in with something generic.
- Leading prompts. Asking for "the research showing" a point invites the model to supply some.
- Long outputs. More tokens mean more chances for a wrong turn that the rest then supports.
A hotel's newsletter draft invents an award
Corrabeg House Hotel, an invented country hotel, asks a chat tool for a 300-word spring newsletter "mentioning our recent recognition". The prompt includes no details of any recognition. The draft says the hotel was "named one of the region's top 10 spa retreats" by a travel magazine and quotes 2 sentences from a guest review. Neither the ranking nor the review exists. The model had been asked for recognition, and a ranking with a number and a publication is what recognition usually looks like in text. The marketing manager reruns the prompt with the hotel's actual 3 pieces of recent coverage pasted in, plus the instruction to use only those and to write "no source" for anything else. The second draft mentions only the real coverage, and the manager checks each line against the pasted material before sending.
What the model can and cannot see
A model only works with what is in its input at the moment it writes, plus what it absorbed in training. That boundary explains a large share of disappointing results.
It can see the prompt, the tool's hidden instructions, the earlier conversation up to a length limit, and any documents or passages the tool has added. It cannot see the team's shared drive, the CRM, the live website, the analytics account or the brand guidelines unless one of them has been placed in the input. It cannot see what happened after its training text was collected. It has no memory of other conversations unless the tool has a feature that stores and reinserts them. And in a long conversation or a long document, material can fall outside the limit of what the model takes in at once, or be given less weight than material near the end.
This is why the most effective single improvement to most marketing prompts is to supply the material: the product sheet, the approved claims, the audience description, the previous version. Tools that do this automatically from a document store are explained in how grounding and retrieval work.
What it means for you
- Give it the facts. Anything that must be true belongs in the prompt or the attached material, with an instruction to use only that and to say when something is missing.
- Use it for what prediction is good at. Structure, phrasing, variations, summaries of supplied material and rewriting are strong; recalling specific, recent or company-only facts is weak.
- Check specifics, not style. Review time goes on figures, names, dates, quotes, sources and product claims. A draft that reads well has told you nothing about whether it is accurate.
- Expect variation. Judge a prompt on several runs and several inputs, as the foundations hub recommends, and measure the share of usable output as a KPIA metric tied directly to an objective. for the task rather than trusting one good result.
- Keep the expertise human. Content that has to show real experience, which search guidance sums up as E-E-A-TExperience, Expertise, Authoritativeness, Trust., cannot get it from a model that has none.
Common misconceptions
- "It looked it up." Unless the tool has a search or retrieval step, the model wrote from learned patterns. A confident reference is not evidence that anything was checked.
- "It knows our company." It knows what was in its training text, which rarely includes a company's current products, prices or policies.
- "A better-written prompt stops hallucination." Clear prompts help, but the dependable fix is supplying the facts and checking the output against them.
- "If it is wrong, it will sound unsure." Tone and accuracy are produced independently; false statements come out as fluently as true ones.
- "It understands the brief like a person would." It responds to the text it is given, and fills gaps in that text with what is typical, which is how generic copy happens.
- "The same prompt gives the same answer." Built-in randomness means outputs vary, and one good run proves little about the next.
