What programmatic pages are
A programmatic page set has three parts: a pattern of queries that share a shape (a fixed phrase plus a variable, such as "[tool] integration with [tool]" or "[job title] salary in [city]"), a dataset with one row per value of the variable, and a template that turns each row into a page. Catalogues, store finders, property listings and directories are older forms of the same idea, and many of them are the most useful pages on the sites that run them.
The shape suits long-tail queriesA specific, usually longer query searched rarely on its own but often as a group.: each one small and specific, numerous enough together to matter. It does not suit queries where the searcher needs judgement, explanation or experience that a dataset cannot supply. A template can tell someone the typical pay for a role in a city; it cannot tell them how to negotiate it.
Two neighbouring patterns have owners elsewhere on this site. Pages for real branches and premises are covered in location pages and listings at scale, and the filter combinations a shop's navigation generates are a crawl-control problem handled in the faceted navigation guide. This guide covers every other template-and-dataset pattern.
The test every templated page must pass
Each page must pass two tests, and the set passes only if its pages pass individually:
- A distinct need. A real searcher typing that combination wants something different from someone typing a neighbouring combination. Pay for one role in two different cities is two needs if the figures differ and people look for both; "accountant salary" and "accountant salaries" are one need.
- Different substance. The page shows information that differs from its siblings in ways that matter to that searcher: different figures, different items, different conditions, different answers. Swapping the variable into otherwise identical text is not different substance.
Google's helpful content guidance sharpens both. It asks whether content provides original information, research or analysis, and whether it adds substantial value compared with other pages in search results. Among its warning signs of content made for search engines first, it asks whether you are producing lots of content on many topics in the hope that some of it performs, and whether you are using extensive automation to do so. That last question describes programmatic sets sized by how many combinations exist rather than by how many deserve a page.
A useful exercise before building anything: generate three pages for random combinations, cover the variable on each, and ask whether a reader could tell them apart from the substance alone. If not, the set fails, however polished the template.
Finding a pattern with real demand
Start from searches, not from the dataset. A spreadsheet of every city multiplied by every product produces combinations; it does not show that anyone is looking for them.
- Confirm the pattern exists in search. Keyword tools and Search Console show whether queries of the shape occur, with a Search IntentThe goal behind a search query: to learn, to find a site, to compare or to act. the template can satisfy. Look at the results pages for a sample: if they show data pages, directories or other templated pages, the format fits; if they show long guides, forums or videos, a template may not satisfy the need.
- Check the spread. Demand is rarely even. Some values of the variable attract many searches, many attract a few, and a long tail has none you can detect. That spread decides how many pages to build, not the size of the dataset.
- Check the business has a reason to answer. A software company with a genuine integration has something to say on an integration page; a page for an integration it does not offer misleads the reader who lands on it.
- Check what already ranks. If the pages ranking for the pattern already show the same public data, a new set needs better or additional data to be worth a searcher's click.
The topic clusters guide applies here too. A templated set is usually a layer of supporting pages beneath a hub that explains the subject, not a substitute for the hub, and the hub is where the editorial judgement the template cannot supply belongs. The academy exercise on planning a templated breed page programme for a dog food maker practises testing a breed and life-stage pattern for real demand and unique data before any page is built, then planning its staged rollout.
The data that makes each page different
The dataset is the product. Templates are cheap; data that differs meaningfully between pages is hard to get, and it is what competitors cannot easily copy.
| Data source | Strength | Watch for |
|---|---|---|
| Your own operational data (prices, availability, usage, outcomes) | Unique to you and naturally different between pages | Thin rows for rare combinations; confidentiality |
| Data users contribute (reviews, submitted figures, ratings) | Unique, and grows over time | Too few contributions per page; moderation and quality |
| Licensed or partner data | Broad coverage quickly | Others may license the same data; terms of use |
| Public datasets | Free and authoritative | Everyone else has them too; little added on their own |
| Data compiled and checked by hand | Accurate and specific | Expensive; needs a maintenance plan |
| Text generated from the row | Fluent prose at almost no cost | Says nothing the row did not; reads the same on every page |
Set a minimum before any page can be published: a number of data points, a set of fields that must be filled, a limit on how old the data may be. The minimum is the mechanism that stops thin combinations from becoming pages. Record for every row whether it meets the minimum, so publication is decided by the data rather than case by case.
Combine sources where you can. Public figures, your own records and a user contribution together give each page something no single source provides.
Then there is text generated from each row by a language model. Google's spam policies give generating many pages with generative tools without adding value for users as an example of scaled content abuse, and the guide to AI in search results covers Google's wider guidance on AI-made content. Generated text that restates the row in sentences adds length, not value.
Designing the template
The template decides what every page shows, so design it with a real row on the screen rather than placeholders, and test it against the best, the worst and the strangest rows in the dataset.
- Lead with the answer. The figure, the list, the compatibility, the price: what the searcher came for, at the top.
- Show the data rather than sentences about the data. Tables, charts and lists carry dataset content better than paragraphs that restate it.
- Add context that varies. Comparisons with neighbouring combinations (this city against the national figure, this role against similar roles), trends over time, the sample behind the figure and the date it was last refreshed.
- Write the fixed parts once, properly. Methodology, definitions and explanations that never vary belong on one page that every templated page links to, not repeated in full on each. The tutorial on building a glossary applies the same test to definitions, keeping most terms on one page with anchors.
- Handle thin rows honestly. If a row falls below the minimum after publication, the page should show less or be withdrawn, not pad itself with generic text.
- Make titles and headings specific. They come from the row and must read naturally for every combination, including awkward values and long names.
- Give the next step. Links to the broader page, the neighbouring combinations and the business's own offer where it genuinely fits the searcher.
Rolling out in stages
Publish a programmatic set in tranches, starting with the combinations that have the most data and the clearest demand.
- Pilot. A small set of the strongest pages, indexable and linked. Read them all by hand before release.
- Measure. Over the following weeks, check in Search Console whether the pilot pages are crawled and indexed and whether they earn impressions and clicks for the intended queries, and check in analytics whether visitors use them: scrolling to the data, following the next step, returning.
- Decide. If the pilot pages are indexed and used, release the next tranche. If many sit discovered but not indexed, or are indexed and ignored, fix the template or the data before releasing more.
- Expand by data, not by count. Each tranche is the next set of rows that meets the minimum, not a target number of pages.
Releasing every combination at once produces a large set of URLs for crawlers to work through and makes it hard to see which pages earn their place. If pages pile up as discovered but not yet indexed, the troubleshooting page for that status covers the causes, including URLs produced faster than the site gives crawlers reasons to fetch them.
Indexing and internal links at scale
Every row ends up in one of three states, and the decision should be recorded in the dataset itself:
| State | Treatment |
|---|---|
| Meets the minimum and matches a search need | Indexable page, in the sitemap, linked from its hub and its neighbours |
| Useful to visitors on the site but not as a search result | Page carrying noindexA robots directive telling search engines not to show a page in their results., linked for visitors where it helps |
| Below the minimum | No page; the combination is served by a broader page |
Google's noindex documentation explains that a noindex rule, in a meta tag or an HTTP response header, keeps a page out of results once Google crawls it, and that the page must not be blocked by robots.txt or the rule cannot be seen. Its large-site guide to Crawl BudgetHow much a search engine crawls a site. adds that noindexed pages are still requested before being dropped, so noindex is no way to save crawling, while also saying that most sites do not need to worry about crawl budget at all. For a set of thousands of pages both points apply: do not generate pages that should not exist, and keep the indexable set to rows that pass the test.
Internal linking holds the set together. A hub page for each dimension (all roles, all cities) lists its children with plain, crawlable links; each page links up to its hubs and across to its nearest neighbours (similar roles in the same city, the same role in nearby cities); and editorial pages link into the set wherever a reader would want the detail. A sound Internal LinkA link from one page to another page on the same website. structure matters more at scale than anywhere else, because templated pages that nothing links to give crawlers little reason to fetch them.
Retire pages that later fall below the minimum by deliberate decision: redirect to the broader page where it serves the same need, or return a not-found status and remove the links and the sitemap entry.
Where it crosses into spam
Google's spam policies describe several patterns a programmatic set can fall into:
- Scaled content abuse. Many pages generated primarily to manipulate rankings rather than help users, typically unoriginal and of little value, however they are made. The examples include generating many pages with automated tools without adding value, scraping feeds or search results to build pages, and stitching together content from other pages without adding value.
- Doorways. Sites or pages created to rank for specific, similar queries that lead users to essentially the same destination, including pages aimed at regions or cities that funnel users to one page, and substantially similar pages that are closer to search results than to a clearly defined, browsable hierarchy.
- Scraping. Content copied from other sites, including copies slightly altered or reworded by automation, republished without original content or added value.
- Keyword stuffing. Among its examples, blocks of text listing the cities and regions a page is trying to rank for.
The guide to Google's spam policies walks through the whole policy page. For a programmatic set the line falls where this guide's test falls. If each page answers a distinct need with substance its siblings lack, the set is a catalogue. If the pages exist because the combinations exist, it is the pattern the policies describe, and the policies page states that sites violating them may rank lower or not appear in results at all.
A salary site that indexed only the combinations it could support
Wrenfiddle Salaries, an invented salary-data site, collects pay figures submitted by employees and licensed from two payroll partners. Its plan for job title by city pages starts from 600 titles and 80 cities: 48,000 possible combinations. A minimum of 30 data points no older than a year leaves 4,100 combinations to publish as indexable pages, each with the median and range, a comparison with the national figure for the role and with nearby cities, the sample size and the date. Another 9,000 combinations have between 10 and 29 data points; they get noindexed pages marked "limited data", linked from the city overviews for visitors who want them. The remaining 34,900 get no page; their figures feed the national role pages. A pilot of 500 indexable pages goes live first. Six weeks later Search Console shows 430 of them indexed and 380 earning impressions, so the other 3,600 follow in three tranches. Six months after launch, 3,700 of the 4,100 indexable pages are indexed. Each quarter the team promotes combinations that have reached the minimum and folds pages that have fallen below it back into their role pages with redirects.
Common mistakes
- Sizing the set by the dataset. The number of combinations is not the number of needs.
- Designing the template with placeholders. Problems show only when real rows, especially thin and awkward ones, are put into it.
- Letting generated prose stand in for data. Fluent paragraphs that restate a row add length, not substance.
- Publishing every combination at once. Nothing can be learned, and a large backlog of uncrawled URLs is likely.
- Using noindex as a crawl saver. Noindexed pages are still requested; pages that should not exist should not be generated.
- No hubs. Templated pages reachable only through site search or the sitemap are left without the links that give crawlers a reason to visit.
- No maintenance plan. Data ages, rows thin out, and the set slowly becomes the stale catalogue nobody trusts.
- Pages for things the business does not offer. An integration page for an integration that does not exist is a page built only to rank.
