Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

SEO › SEO Content & Strategy

Programmatic SEO Without Thin Pages

How to build many pages from one template and a dataset without producing thin or doorway pages: demand, unique data, template design, rollout and indexing.
Guideadvanced11 min readUpdated

What programmatic pages are

A programmatic page set has three parts: a pattern of queries that share a shape (a fixed phrase plus a variable, such as "[tool] integration with [tool]" or "[job title] salary in [city]"), a dataset with one row per value of the variable, and a template that turns each row into a page. Catalogues, store finders, property listings and directories are older forms of the same idea, and many of them are the most useful pages on the sites that run them.

The shape suits long-tail queries: each one small and specific, numerous enough together to matter. It does not suit queries where the searcher needs judgement, explanation or experience that a dataset cannot supply. A template can tell someone the typical pay for a role in a city; it cannot tell them how to negotiate it.

Two neighbouring patterns have owners elsewhere on this site. Pages for real branches and premises are covered in location pages and listings at scale, and the filter combinations a shop's navigation generates are a crawl-control problem handled in the faceted navigation guide. This guide covers every other template-and-dataset pattern.

The test every templated page must pass

Each page must pass two tests, and the set passes only if its pages pass individually:

  1. A distinct need. A real searcher typing that combination wants something different from someone typing a neighbouring combination. Pay for one role in two different cities is two needs if the figures differ and people look for both; "accountant salary" and "accountant salaries" are one need.
  2. Different substance. The page shows information that differs from its siblings in ways that matter to that searcher: different figures, different items, different conditions, different answers. Swapping the variable into otherwise identical text is not different substance.

Google's helpful content guidance sharpens both. It asks whether content provides original information, research or analysis, and whether it adds substantial value compared with other pages in search results. Among its warning signs of content made for search engines first, it asks whether you are producing lots of content on many topics in the hope that some of it performs, and whether you are using extensive automation to do so. That last question describes programmatic sets sized by how many combinations exist rather than by how many deserve a page.

A useful exercise before building anything: generate three pages for random combinations, cover the variable on each, and ask whether a reader could tell them apart from the substance alone. If not, the set fails, however polished the template.

Finding a pattern with real demand

Start from searches, not from the dataset. A spreadsheet of every city multiplied by every product produces combinations; it does not show that anyone is looking for them.

  • Confirm the pattern exists in search. Keyword tools and Search Console show whether queries of the shape occur, with a Search Intent the template can satisfy. Look at the results pages for a sample: if they show data pages, directories or other templated pages, the format fits; if they show long guides, forums or videos, a template may not satisfy the need.
  • Check the spread. Demand is rarely even. Some values of the variable attract many searches, many attract a few, and a long tail has none you can detect. That spread decides how many pages to build, not the size of the dataset.
  • Check the business has a reason to answer. A software company with a genuine integration has something to say on an integration page; a page for an integration it does not offer misleads the reader who lands on it.
  • Check what already ranks. If the pages ranking for the pattern already show the same public data, a new set needs better or additional data to be worth a searcher's click.

The topic clusters guide applies here too. A templated set is usually a layer of supporting pages beneath a hub that explains the subject, not a substitute for the hub, and the hub is where the editorial judgement the template cannot supply belongs. The academy exercise on planning a templated breed page programme for a dog food maker practises testing a breed and life-stage pattern for real demand and unique data before any page is built, then planning its staged rollout.

The data that makes each page different

The dataset is the product. Templates are cheap; data that differs meaningfully between pages is hard to get, and it is what competitors cannot easily copy.

Data sourceStrengthWatch for
Your own operational data (prices, availability, usage, outcomes)Unique to you and naturally different between pagesThin rows for rare combinations; confidentiality
Data users contribute (reviews, submitted figures, ratings)Unique, and grows over timeToo few contributions per page; moderation and quality
Licensed or partner dataBroad coverage quicklyOthers may license the same data; terms of use
Public datasetsFree and authoritativeEveryone else has them too; little added on their own
Data compiled and checked by handAccurate and specificExpensive; needs a maintenance plan
Text generated from the rowFluent prose at almost no costSays nothing the row did not; reads the same on every page

Set a minimum before any page can be published: a number of data points, a set of fields that must be filled, a limit on how old the data may be. The minimum is the mechanism that stops thin combinations from becoming pages. Record for every row whether it meets the minimum, so publication is decided by the data rather than case by case.

Combine sources where you can. Public figures, your own records and a user contribution together give each page something no single source provides.

Then there is text generated from each row by a language model. Google's spam policies give generating many pages with generative tools without adding value for users as an example of scaled content abuse, and the guide to AI in search results covers Google's wider guidance on AI-made content. Generated text that restates the row in sentences adds length, not value.

Designing the template

The template decides what every page shows, so design it with a real row on the screen rather than placeholders, and test it against the best, the worst and the strangest rows in the dataset.

  • Lead with the answer. The figure, the list, the compatibility, the price: what the searcher came for, at the top.
  • Show the data rather than sentences about the data. Tables, charts and lists carry dataset content better than paragraphs that restate it.
  • Add context that varies. Comparisons with neighbouring combinations (this city against the national figure, this role against similar roles), trends over time, the sample behind the figure and the date it was last refreshed.
  • Write the fixed parts once, properly. Methodology, definitions and explanations that never vary belong on one page that every templated page links to, not repeated in full on each. The tutorial on building a glossary applies the same test to definitions, keeping most terms on one page with anchors.
  • Handle thin rows honestly. If a row falls below the minimum after publication, the page should show less or be withdrawn, not pad itself with generic text.
  • Make titles and headings specific. They come from the row and must read naturally for every combination, including awkward values and long names.
  • Give the next step. Links to the broader page, the neighbouring combinations and the business's own offer where it genuinely fits the searcher.

Rolling out in stages

Publish a programmatic set in tranches, starting with the combinations that have the most data and the clearest demand.

  1. Pilot. A small set of the strongest pages, indexable and linked. Read them all by hand before release.
  2. Measure. Over the following weeks, check in Search Console whether the pilot pages are crawled and indexed and whether they earn impressions and clicks for the intended queries, and check in analytics whether visitors use them: scrolling to the data, following the next step, returning.
  3. Decide. If the pilot pages are indexed and used, release the next tranche. If many sit discovered but not indexed, or are indexed and ignored, fix the template or the data before releasing more.
  4. Expand by data, not by count. Each tranche is the next set of rows that meets the minimum, not a target number of pages.

Releasing every combination at once produces a large set of URLs for crawlers to work through and makes it hard to see which pages earn their place. If pages pile up as discovered but not yet indexed, the troubleshooting page for that status covers the causes, including URLs produced faster than the site gives crawlers reasons to fetch them.

Every row ends up in one of three states, and the decision should be recorded in the dataset itself:

StateTreatment
Meets the minimum and matches a search needIndexable page, in the sitemap, linked from its hub and its neighbours
Useful to visitors on the site but not as a search resultPage carrying noindex, linked for visitors where it helps
Below the minimumNo page; the combination is served by a broader page

Google's noindex documentation explains that a noindex rule, in a meta tag or an HTTP response header, keeps a page out of results once Google crawls it, and that the page must not be blocked by robots.txt or the rule cannot be seen. Its large-site guide to Crawl Budget adds that noindexed pages are still requested before being dropped, so noindex is no way to save crawling, while also saying that most sites do not need to worry about crawl budget at all. For a set of thousands of pages both points apply: do not generate pages that should not exist, and keep the indexable set to rows that pass the test.

Internal linking holds the set together. A hub page for each dimension (all roles, all cities) lists its children with plain, crawlable links; each page links up to its hubs and across to its nearest neighbours (similar roles in the same city, the same role in nearby cities); and editorial pages link into the set wherever a reader would want the detail. A sound Internal Link structure matters more at scale than anywhere else, because templated pages that nothing links to give crawlers little reason to fetch them.

Retire pages that later fall below the minimum by deliberate decision: redirect to the broader page where it serves the same need, or return a not-found status and remove the links and the sitemap entry.

Where it crosses into spam

Google's spam policies describe several patterns a programmatic set can fall into:

  • Scaled content abuse. Many pages generated primarily to manipulate rankings rather than help users, typically unoriginal and of little value, however they are made. The examples include generating many pages with automated tools without adding value, scraping feeds or search results to build pages, and stitching together content from other pages without adding value.
  • Doorways. Sites or pages created to rank for specific, similar queries that lead users to essentially the same destination, including pages aimed at regions or cities that funnel users to one page, and substantially similar pages that are closer to search results than to a clearly defined, browsable hierarchy.
  • Scraping. Content copied from other sites, including copies slightly altered or reworded by automation, republished without original content or added value.
  • Keyword stuffing. Among its examples, blocks of text listing the cities and regions a page is trying to rank for.

The guide to Google's spam policies walks through the whole policy page. For a programmatic set the line falls where this guide's test falls. If each page answers a distinct need with substance its siblings lack, the set is a catalogue. If the pages exist because the combinations exist, it is the pattern the policies describe, and the policies page states that sites violating them may rank lower or not appear in results at all.

Illustrative example — not a real result

A salary site that indexed only the combinations it could support

Wrenfiddle Salaries, an invented salary-data site, collects pay figures submitted by employees and licensed from two payroll partners. Its plan for job title by city pages starts from 600 titles and 80 cities: 48,000 possible combinations. A minimum of 30 data points no older than a year leaves 4,100 combinations to publish as indexable pages, each with the median and range, a comparison with the national figure for the role and with nearby cities, the sample size and the date. Another 9,000 combinations have between 10 and 29 data points; they get noindexed pages marked "limited data", linked from the city overviews for visitors who want them. The remaining 34,900 get no page; their figures feed the national role pages. A pilot of 500 indexable pages goes live first. Six weeks later Search Console shows 430 of them indexed and 380 earning impressions, so the other 3,600 follow in three tranches. Six months after launch, 3,700 of the 4,100 indexable pages are indexed. Each quarter the team promotes combinations that have reached the minimum and folds pages that have fallen below it back into their role pages with redirects.

Common mistakes

  • Sizing the set by the dataset. The number of combinations is not the number of needs.
  • Designing the template with placeholders. Problems show only when real rows, especially thin and awkward ones, are put into it.
  • Letting generated prose stand in for data. Fluent paragraphs that restate a row add length, not substance.
  • Publishing every combination at once. Nothing can be learned, and a large backlog of uncrawled URLs is likely.
  • Using noindex as a crawl saver. Noindexed pages are still requested; pages that should not exist should not be generated.
  • No hubs. Templated pages reachable only through site search or the sitemap are left without the links that give crawlers a reason to visit.
  • No maintenance plan. Data ages, rows thin out, and the set slowly becomes the stale catalogue nobody trusts.
  • Pages for things the business does not offer. An integration page for an integration that does not exist is a page built only to rank.

Frequently asked questions

Is programmatic SEO against Google's guidelines?

Building pages from a template and data is not in itself a violation; product catalogues and directories have always worked that way. Google's spam policies target scaled content abuse, many pages generated mainly to manipulate rankings rather than to help users, and doorways, similar pages built to rank for similar queries that lead to the same place. Whether a programmatic set is on the wrong side depends on whether each page gives a searcher something its siblings do not, not on how the pages were produced.

How many pages should a programmatic launch start with?

As few as will show whether the pattern works: a sample of the strongest combinations, those with the most data and the clearest demand, published as indexable pages while the rest are held back. Check whether they are crawled, indexed, found and used before releasing the next tranche. Publishing every combination at once makes it hard to see which pages earn their place, and can leave large numbers of pages waiting to be crawled.

Should thin combinations be noindexed or not created at all?

If a combination has nothing distinct to show, the better answer is usually not to create a page for it and to serve it from a broader page instead. Noindex keeps a page out of results, but Google's crawl budget guide notes that noindexed pages are still requested before they are dropped, so a large noindexed set still costs crawling. Noindex suits pages that are useful to visitors already on the site but are not worth showing as search results.

Terms used on this page

Frameworks

Where this page is used

Part of these learning paths

Sources

  1. Spam policies for Google web search — Google Search Central
  2. Creating helpful, reliable, people-first content — Google Search Central
  3. Large site owner's guide to managing your crawl budget — Google Search Central
  4. Block Search indexing with noindex — Google Search Central

Related