Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › AI Applications in Marketing

AI for Keyword Research and Clustering

Where AI helps keyword research, from seed ideas and intent labels to grouping, and where it fails: invented volumes, stale data and groups results contradict.
Guideintermediate9 min readUpdated

What AI can and cannot know about search demand

A language model learns patterns of language from the text it was trained on. That makes it good at the vocabulary around a topic: the names people give a problem, the questions they ask, the comparisons they make, the jargon experts use and the plainer words beginners use instead. Google's SEO Starter Guide encourages site owners to think about the words people would use to look for their content, including the different words used by people who know a topic well and people who are new to it. Producing that range of wording is the job a model does well.

What a model does not have is search data. It has not seen the query logs, it does not know how often a phrase is searched this month, and it cannot see the results page for any query unless a connected tool fetches it. Its training text also has a cut-off, so new products, new terms and seasonal shifts after that date are missing. Ask it for a Search Volume figure, a difficulty score or "the top ranking pages" and it will answer anyway, with numbers and URLs that look right and come from nowhere.

QuestionCan a model answer it?Where the answer actually comes from
What words might people use for this?Yes, as a list of candidatesModel draft, then checked against real queries
What do people ask around this topic?Partly, as candidatesSearch Console queries, related searches, site search, support logs
How many people search for this?NoSearch Console, the keyword planner, keyword tools (all estimates)
What intent does this query have?A draft label from wording aloneThe live results page decides
Which queries should share a page?A draft grouping by meaningResults-page overlap confirms or splits
Is this query rising or falling?NoTool trend data and the site's own query history
Which pages rank for it today?No, unless a tool fetches the resultsThe results page itself

Some tools connect a model to keyword databases or fetch live results. In that case the data comes from the connected source, and the checks below still apply to whatever the model does with it: summarising, filtering and grouping are all places where a model can drop, merge or misreport rows. How AI-generated answers on the results page change what a ranking is worth is a separate question, covered in the AI in search results guide.

Generating seeds and modifiers

Seed generation is the safest application, because nothing a model suggests here is used without being checked against demand data. Give it the product, the audience, the problems the product solves and a sample of real queries from Search Console, and ask for candidate seeds grouped by theme, with the customer's wording rather than the company's. Ask separately for the words a beginner would use and the words an expert would use, since those lists rarely overlap.

Modifiers are where models add the most. Ask for the qualifiers people attach to a head term: comparisons, problems, use cases, audiences, locations, price levels, formats, years and question forms. These expand a short seed list into the Long-Tail Keyword candidates that a site can realistically win and that a keyword tool can then confirm or reject. A model can also read customer material the team supplies, such as support tickets, reviews and sales call notes, and pull out the phrases customers actually use, which are often missing from every keyword tool because they are too specific to show volume.

Two cautions apply. A model fills gaps with plausible phrases that nobody searches, so every candidate needs a demand check before it earns a place in the plan. And a model trained on general text drifts towards generic vocabulary; the industry's own words and the customer's idiosyncratic ones come from supplied material, not from the model's memory.

Labelling intent at scale

Labelling Search Intent across a long list is repetitive, rule-based work, which suits a model when the rules are set by a person. The intent classification guide describes the categories, the modifier rules and the results-page signals; those become the model's instructions. Give it the category definitions, the modifier rules, a set of labelled examples including the hard cases, and an explicit "unclear" option. Ask for a label and a one-line reason per query, in a table rather than prose.

The reason column is what makes the output checkable. A label of "commercial" with the reason "contains 'best'" can be verified in a second; a label with no reason has to be worked out again from scratch. Keep the instructions fixed for the whole list, so the labels are comparable, and run the list in batches small enough that rows are not dropped.

The limit is that intent is decided by the results page, not by wording. A query with no modifier at all may show shopping results, a definition, local listings or a mix, and the mix changes over time. A model labelling from words alone will be confidently wrong on exactly the ambiguous head terms that matter most. Treat its labels as a draft: accept them for clear modifier cases after a sample check, and send head terms, mixed queries and anything labelled unclear to a person who looks at the live results.

Grouping by meaning versus by results

The keyword clustering guide describes two ways to group queries: by meaning, and by how much the search results for each query overlap. A model groups by meaning, whether by reading the queries or through similarity scores computed from the text, the mechanism the embeddings explainer describes. That is fast and cheap, and it gives a sensible first draft on a long list.

It fails in two predictable directions. It merges queries that share words but differ in intent, such as a product name searched by people who want to buy it and by people who want to fix it. And it separates queries the engine treats as one, such as an abbreviation and its full form, or a brand's name and a generic description of its product. Either error produces a keyword map with the wrong number of pages: merged groups leave demand unserved, and split groups create pages that compete with each other, the Keyword Cannibalisation problem the clustering guide warns about.

The practical sequence follows the clustering guide: a model proposes groups by meaning, intent labels act as a hard boundary so that different intents never share a group, and the results-page comparison confirms or splits the groups that carry real value. Ask the model to give each group a name, a proposed primary query and a short note on what the group asks for; those notes speed up the results-page check.

Checking AI output against real data

Every AI suggestion in keyword research is a hypothesis about demand or about the engine, and each kind has a specific check.

Candidates against demand. Run every seed and modifier through the data sources the keyword research section describes: Search Console for queries the site already appears for, the keyword planner and keyword tools for estimates, related searches for phrasing. Candidates with no evidence of demand are parked, not deleted, since some will show up later in Search Console.

Labels against results. Check a random sample of intent labels against the live results page, plus every head term and every query labelled unclear. If disagreement clusters on one category, the definition or the examples are unclear; fix them and re-run the affected rows.

Groups against overlap. Compare the results for the primary query and the members of each valuable group. A member whose results look different leaves the group, and two groups whose results look the same merge.

Rows against the input. Count rows in and rows out. Models drop, duplicate and silently merge rows in long lists, and a missing query is invisible unless someone checks the totals.

Numbers against their source. Any figure in the output, whether a volume, a difficulty score or a trend, must trace to a named data source. A number the model produced on its own is deleted.

A workflow that keeps the data honest

StageModel's jobData's jobPerson's job
SeedsCandidate terms in customer and expert vocabularyReal queries from Search Console as inputSupply product, audience and customer material
ModifiersQualifiers and question forms around each seedDemand estimates from keyword toolsRemove irrelevant and off-brand candidates
Customer phrasesExtract phrases from tickets, reviews and call notesSite search and support logs as inputChoose which phrases are worth testing
IntentDraft labels with a reason per queryThe live results pageSet the rules; check samples, head terms and unclear rows
GroupingDraft groups by meaning, with names and notesResults-page overlapConfirm or split valuable groups
MappingDraft group summaries for the mapCurrent rankings and pagesAssign owner URLs in the keyword map
ReviewRe-run on new queries each cycleFresh Search Console dataDecide what changes in the map

The pattern in every row is the same: the model drafts, data decides, and a person signs off. The SEO strategy framework places keyword research inside the wider plan; nothing in this workflow changes that plan's order, it only speeds up the drafting steps.

Illustrative example — not a real result

Expanding and grouping a keyword list for a bike parts shop

Pellwick Cycles, an invented online shop selling bicycle parts, starts with 40 seed queries from Search Console. A model expands them to 620 candidates using supplied support tickets and a modifier prompt. A keyword tool finds estimated demand for 214 of them; the rest are parked. The model labels intent for the 214 with a reason per row and groups them into 38 groups by meaning. A person checks 30 random labels against the live results and agrees with 25; four of the five disagreements are head terms such as "tubeless tyres", where the results mix guides and product listings. Checking the 12 most valuable groups against results overlap splits 3 of them, including one that had merged "chain wear" (how to measure it) with "chain wear tool" (a product). The final map has 41 groups. The row count check finds the model had dropped 9 queries from its grouping output; they are added back by hand.

Common mistakes

  • Taking volumes from the model. A figure with no data source behind it is invented, however precise it looks.
  • Treating the candidate list as the keyword list. Model suggestions are hypotheses until demand data supports them.
  • Labelling intent from words alone. The results page decides intent, and it changes.
  • Accepting meaning-based groups without a results check. Shared words do not mean a shared page.
  • Letting groups cross intent boundaries. Different intents never share a page, whatever the wording.
  • Not counting rows. Long lists lose rows silently in model output.
  • Asking for generic vocabulary. The customer's own words come from supplied material, not the model's memory.
  • Changing the labelling instructions mid-list. Labels are only comparable when the rules stay fixed.

Frequently asked questions

Can I trust search volumes that an AI assistant gives me?

No. A general language model has no access to search data unless a tool connects it to a keyword data source, and even then the figure is the data source's estimate, not the model's. Asked for a volume without that connection, a model produces a number that looks plausible and rests on nothing. Take volumes from Search Console, the keyword planner or a keyword tool, and treat even those as relative estimates.

Is AI keyword clustering as good as clustering by search results?

They answer different questions. Grouping by meaning shows which queries ask for the same thing in different words; grouping by results overlap shows which queries the search engine actually answers with the same pages. Meaning-based groups are a fast first draft, and they go wrong exactly where it matters most, on queries that share words but differ in intent. Use a model to propose groups and the results page to confirm or split the valuable ones.

Should I use AI to label search intent for thousands of keywords?

It is a reasonable use, provided the categories and rules are yours, the model returns a label with a short reason for each query, and a person checks a sample against the live results page. Intent depends on what the engine currently shows, which a model cannot see, so treat its labels as a draft that the results-page check can overturn.

Terms used on this page

Frameworks

Where this page is used

Sources

  1. SEO Starter Guide — Google Search Central

Related