What AI can and cannot know about search demand
A language model learns patterns of language from the text it was trained on. That makes it good at the vocabulary around a topic: the names people give a problem, the questions they ask, the comparisons they make, the jargon experts use and the plainer words beginners use instead. Google's SEO Starter Guide encourages site owners to think about the words people would use to look for their content, including the different words used by people who know a topic well and people who are new to it. Producing that range of wording is the job a model does well.
What a model does not have is search data. It has not seen the query logs, it does not know how often a phrase is searched this month, and it cannot see the results page for any query unless a connected tool fetches it. Its training text also has a cut-off, so new products, new terms and seasonal shifts after that date are missing. Ask it for a Search VolumeThe estimated number of times a query is searched in a period, usually per month. figure, a difficulty score or "the top ranking pages" and it will answer anyway, with numbers and URLs that look right and come from nowhere.
| Question | Can a model answer it? | Where the answer actually comes from |
|---|---|---|
| What words might people use for this? | Yes, as a list of candidates | Model draft, then checked against real queries |
| What do people ask around this topic? | Partly, as candidates | Search Console queries, related searches, site search, support logs |
| How many people search for this? | No | Search Console, the keyword planner, keyword tools (all estimates) |
| What intent does this query have? | A draft label from wording alone | The live results page decides |
| Which queries should share a page? | A draft grouping by meaning | Results-page overlap confirms or splits |
| Is this query rising or falling? | No | Tool trend data and the site's own query history |
| Which pages rank for it today? | No, unless a tool fetches the results | The results page itself |
Some tools connect a model to keyword databases or fetch live results. In that case the data comes from the connected source, and the checks below still apply to whatever the model does with it: summarising, filtering and grouping are all places where a model can drop, merge or misreport rows. How AI-generated answers on the results page change what a ranking is worth is a separate question, covered in the AI in search results guide.
Generating seeds and modifiers
Seed generation is the safest application, because nothing a model suggests here is used without being checked against demand data. Give it the product, the audience, the problems the product solves and a sample of real queries from Search Console, and ask for candidate seeds grouped by theme, with the customer's wording rather than the company's. Ask separately for the words a beginner would use and the words an expert would use, since those lists rarely overlap.
Modifiers are where models add the most. Ask for the qualifiers people attach to a head term: comparisons, problems, use cases, audiences, locations, price levels, formats, years and question forms. These expand a short seed list into the Long-Tail KeywordA specific, usually longer query searched rarely on its own but often as a group. candidates that a site can realistically win and that a keyword tool can then confirm or reject. A model can also read customer material the team supplies, such as support tickets, reviews and sales call notes, and pull out the phrases customers actually use, which are often missing from every keyword tool because they are too specific to show volume.
Two cautions apply. A model fills gaps with plausible phrases that nobody searches, so every candidate needs a demand check before it earns a place in the plan. And a model trained on general text drifts towards generic vocabulary; the industry's own words and the customer's idiosyncratic ones come from supplied material, not from the model's memory.
Labelling intent at scale
Labelling Search IntentThe goal behind a search query: to learn, to find a site, to compare or to act. across a long list is repetitive, rule-based work, which suits a model when the rules are set by a person. The intent classification guide describes the categories, the modifier rules and the results-page signals; those become the model's instructions. Give it the category definitions, the modifier rules, a set of labelled examples including the hard cases, and an explicit "unclear" option. Ask for a label and a one-line reason per query, in a table rather than prose.
The reason column is what makes the output checkable. A label of "commercial" with the reason "contains 'best'" can be verified in a second; a label with no reason has to be worked out again from scratch. Keep the instructions fixed for the whole list, so the labels are comparable, and run the list in batches small enough that rows are not dropped.
The limit is that intent is decided by the results page, not by wording. A query with no modifier at all may show shopping results, a definition, local listings or a mix, and the mix changes over time. A model labelling from words alone will be confidently wrong on exactly the ambiguous head terms that matter most. Treat its labels as a draft: accept them for clear modifier cases after a sample check, and send head terms, mixed queries and anything labelled unclear to a person who looks at the live results.
Grouping by meaning versus by results
The keyword clustering guide describes two ways to group queries: by meaning, and by how much the search results for each query overlap. A model groups by meaning, whether by reading the queries or through similarity scores computed from the text, the mechanism the embeddings explainer describes. That is fast and cheap, and it gives a sensible first draft on a long list.
It fails in two predictable directions. It merges queries that share words but differ in intent, such as a product name searched by people who want to buy it and by people who want to fix it. And it separates queries the engine treats as one, such as an abbreviation and its full form, or a brand's name and a generic description of its product. Either error produces a keyword map with the wrong number of pages: merged groups leave demand unserved, and split groups create pages that compete with each other, the Keyword CannibalisationTwo or more pages on one site competing for the same query and intent, splitting signals. problem the clustering guide warns about.
The practical sequence follows the clustering guide: a model proposes groups by meaning, intent labels act as a hard boundary so that different intents never share a group, and the results-page comparison confirms or splits the groups that carry real value. Ask the model to give each group a name, a proposed primary query and a short note on what the group asks for; those notes speed up the results-page check.
Checking AI output against real data
Every AI suggestion in keyword research is a hypothesis about demand or about the engine, and each kind has a specific check.
Candidates against demand. Run every seed and modifier through the data sources the keyword research section describes: Search Console for queries the site already appears for, the keyword planner and keyword tools for estimates, related searches for phrasing. Candidates with no evidence of demand are parked, not deleted, since some will show up later in Search Console.
Labels against results. Check a random sample of intent labels against the live results page, plus every head term and every query labelled unclear. If disagreement clusters on one category, the definition or the examples are unclear; fix them and re-run the affected rows.
Groups against overlap. Compare the results for the primary query and the members of each valuable group. A member whose results look different leaves the group, and two groups whose results look the same merge.
Rows against the input. Count rows in and rows out. Models drop, duplicate and silently merge rows in long lists, and a missing query is invisible unless someone checks the totals.
Numbers against their source. Any figure in the output, whether a volume, a difficulty score or a trend, must trace to a named data source. A number the model produced on its own is deleted.
A workflow that keeps the data honest
| Stage | Model's job | Data's job | Person's job |
|---|---|---|---|
| Seeds | Candidate terms in customer and expert vocabulary | Real queries from Search Console as input | Supply product, audience and customer material |
| Modifiers | Qualifiers and question forms around each seed | Demand estimates from keyword tools | Remove irrelevant and off-brand candidates |
| Customer phrases | Extract phrases from tickets, reviews and call notes | Site search and support logs as input | Choose which phrases are worth testing |
| Intent | Draft labels with a reason per query | The live results page | Set the rules; check samples, head terms and unclear rows |
| Grouping | Draft groups by meaning, with names and notes | Results-page overlap | Confirm or split valuable groups |
| Mapping | Draft group summaries for the map | Current rankings and pages | Assign owner URLs in the keyword map |
| Review | Re-run on new queries each cycle | Fresh Search Console data | Decide what changes in the map |
The pattern in every row is the same: the model drafts, data decides, and a person signs off. The SEO strategy framework places keyword research inside the wider plan; nothing in this workflow changes that plan's order, it only speeds up the drafting steps.
Expanding and grouping a keyword list for a bike parts shop
Pellwick Cycles, an invented online shop selling bicycle parts, starts with 40 seed queries from Search Console. A model expands them to 620 candidates using supplied support tickets and a modifier prompt. A keyword tool finds estimated demand for 214 of them; the rest are parked. The model labels intent for the 214 with a reason per row and groups them into 38 groups by meaning. A person checks 30 random labels against the live results and agrees with 25; four of the five disagreements are head terms such as "tubeless tyres", where the results mix guides and product listings. Checking the 12 most valuable groups against results overlap splits 3 of them, including one that had merged "chain wear" (how to measure it) with "chain wear tool" (a product). The final map has 41 groups. The row count check finds the model had dropped 9 queries from its grouping output; they are added back by hand.
Common mistakes
- Taking volumes from the model. A figure with no data source behind it is invented, however precise it looks.
- Treating the candidate list as the keyword list. Model suggestions are hypotheses until demand data supports them.
- Labelling intent from words alone. The results page decides intent, and it changes.
- Accepting meaning-based groups without a results check. Shared words do not mean a shared page.
- Letting groups cross intent boundaries. Different intents never share a page, whatever the wording.
- Not counting rows. Long lists lose rows silently in model output.
- Asking for generic vocabulary. The customer's own words come from supplied material, not the model's memory.
- Changing the labelling instructions mid-list. Labels are only comparable when the rules stay fixed.
