Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

SEO › E-commerce SEO

Faceted Navigation and Index Control

Which filter combinations deserve indexable URLs and how to keep the rest out of the index and crawl: a decision framework plus the mechanics.
Guideadvanced8 min readUpdated

The problem

Google's faceted navigation documentation describes the issue plainly: filtering and sorting generate many URL variations that show overlapping content, which can lead to the crawler spending its effort on duplicates, slow discovery of new content, and dilution across near-identical pages. For a category with a dozen facets, each with a few values, the combinations run into the millions, and any crawler that follows links will find them.

The symptoms in Search Console (GSC): the Page Indexing report dominated by "duplicate without user-selected canonical", "crawled, currently not indexed" and "discovered, currently not indexed"; crawl stats showing most requests going to parameterised URLs; new products taking weeks to appear. The crawling and indexing guide explains the underlying behaviour and Crawl Budget.

The decision framework

For each facet, and then for the combinations, ask four questions:

  1. Is there search demand for this view? "Women's waterproof hiking boots" is a query; "hiking boots sorted by price, forty per page" is not. The keyword research section provides the data; the keyword map records which URL owns each cluster.
  2. Does the view have enough products to be a useful page? A filter combination that returns two products is a thin page; one that returns a hundred is a category.
  3. Is the view distinct from its parent? A filter that removes a tenth of the products is a near duplicate of the category; one that selects a coherent sub-range is a different page.
  4. Can the view carry its own content? A title, heading, introduction and markup that describe this sub-range specifically. If the template can only swap a word into the parent's title, the view is a filter, not a page.

The outcome is three tiers:

TierExampleTreatment
Indexable landing pageCategory + one high-demand facet (brand, type, a key attribute); occasionally twoClean URL (path or canonical parameter form), self-canonical, own content, in the sitemap, linked from the category and the navigation
Crawlable but not indexableCombinations the crawler will encounter through filter links but that do not deserve rankingCanonical to the nearest indexable page, or noindex where the content differs too much to canonicalise honestly
Neither crawled nor indexedDeep combinations, sorts, page sizes, view modes, session or tracking parametersNot linked with crawlable anchors (script-driven or nofollow), robots rules for the parameter patterns, and canonical as a safety net

The first tier is typically a small number of combinations per category, chosen by demand and reviewed with the keyword map; it is never "every value of every facet".

Designing filter URLs

Google's guidance asks that the engine be able to tell distinct views apart and that URL structure be consistent. Rules that make that true:

  • Indexable combinations get a page-like URL: /hiking-boots/womens/ or /hiking-boots/brand/acme/, with the facet order fixed.
  • Non-indexable filters use parameters in a fixed order with fixed value encoding, so that the same selection always produces the same URL: ?colour=blue&size=9, never ?size=9&colour=blue for the same view. Alphabetical parameter ordering enforced by the platform removes a whole class of duplicates.
  • Sorts, page sizes and view modes either do not change the URL (handled client-side) or use parameters that are canonicalised away and excluded in robots.
  • Session, tracking and internal search parameters never appear in links the crawler can follow.
  • Empty results return a page with a clear message and a noindex, or redirect to the parent; never a soft 404 that looks like a page.
  • Pagination keeps its own parameter and is handled per the category and product guide.

The mechanics

ToolWhat it doesWhere it fits
Canonical tag (Canonical Tag)Suggests the preferred URL among duplicates; passes signalsFilter views that are subsets of an indexable page; Google treats it as a hint and may ignore it where the content differs a lot
noindex metaKeeps the page out of the index while allowing crawlFilter views that are too different from any page to canonicalise honestly, such as combinations with distinct products but no demand
robots.txt rulesPrevents crawling of matching URL patternsSorts, deep combinations, parameters that explode; note blocked URLs can still be indexed if linked, and cannot pass signals
Link attributes (rel="nofollow") on filter linksAsks the crawler not to followDeep filter links; a hint, not a block
Script-driven filtering without URL changeKeeps selections off the URL entirelyFilters that never need a URL; must still expose the indexable tier as real links
SitemapsDeclares the indexable tierOnly tier-one URLs; never filter noise
Internal linkingDecides what the crawler finds firstLink tier-one pages from the category and navigation; do not link deep combinations in crawlable form

Google's documentation retired the parameter-handling tool that once let site owners declare parameters in Search Console, so these on-site mechanics are the whole toolkit. Of them, nofollow on filter links is the weakest on its own, because search engines treat it as a hint rather than a block.

Implementing on a platform

Hosted platforms expose some of these controls and not others, and the platform audit decides what is possible: whether filter URLs can be given canonical tags to the collection, whether chosen combinations can be turned into real collection pages with their own content, whether sort parameters can be excluded in robots, whether filter links can be rendered as buttons rather than anchors. Where a platform cannot canonicalise or noindex filter views, the fallback is to keep filters off the URL and build the indexable tier as separate collections.

Illustrative example — not a real result

An outdoor retailer's boots category

The category has facets for gender, activity, waterproofing, brand, colour, size, price and sort. Query data shows demand for gender, activity and brand, and for the gender-plus-activity pair; nobody searches by colour or price band. The decision: build indexable pages for each gender, each activity, each major brand, and the gender-activity pairs with enough products, each with its own introduction and title, linked from the category and in the sitemap. Waterproofing, colour, size and price remain parameters in fixed order, canonicalised to the nearest indexable page. Sort and page size are handled without changing the URL. Robots rules exclude the parameters that remain. Within a few months the Page Indexing report's duplicate counts fall, crawl requests shift from parameters to products, and the new gender-activity pages begin to appear for the queries they were built for.

Internal search results pages

A store's own search generates URLs (/search?q=...) that are, to the engine, another faceted view: an unbounded set of listing pages with overlapping content. Google's guidance has long advised keeping internal search results pages out of the index. The treatment is the third tier: noindex on the search template, the search parameter excluded in robots.txt, and search links rendered as a form submission rather than crawlable anchors. The one exception is a query so common that it deserves a real landing page — in which case it becomes a tier-one page built as a category, not an indexed search result.

Testing before rollout

Index-control changes on a large store affect thousands of URLs at once, and a mistake — a canonical rule that catches the tier-one pages, a robots rule that blocks the category paths — removes revenue pages from the index for weeks. Before rollout:

  1. List the tier-one URLs and confirm, against the new rules, that each remains self-canonical, indexable and allowed in robots.
  2. Sample the tier-two and tier-three URLs and confirm each resolves to the intended treatment.
  3. Crawl the staging site with the rules applied and compare the indexable URL set to the intended one; the diff is the bug list.
  4. Roll out to one category first on a large site, watch the indexing report and crawl stats for that section, then extend.
  5. Keep the previous rules ready to restore, and annotate the change for the measurement team.

The site migrations guide treats a large index-control change as the migration it is.

Monitoring

After implementation: the Page Indexing report by reason, over time; crawl stats by URL pattern; the share of crawl going to products and tier-one pages versus parameters; the time from product publication to first crawl; and the performance of the tier-one pages for their mapped queries. Server logs give the exact split of crawler requests between products, landing pages and parameters, using the method in the log file analysis tutorial. The technical audit tutorial includes the checks; the SEO page checker reports the canonical and robots directives on any individual URL for spot checks.

Filter and parameter copies frequently show up as duplicates Google has folded into another URL; the diagnosis for duplicates without a user-selected canonical covers how to align the signals.

Why filter views that users rely on take a canonical rather than a redirect is explained in the comparison of canonical tags and 301 redirects.

Filters are only one class of parameter a store emits; the tutorial on auditing URL parameters records sort, tracking, session and search parameters in the same register and gives each class its own rule.

Common mistakes

  • Indexing every facet value. Thousands of thin, duplicative pages.
  • Indexing nothing. The refinements people search for never get a page.
  • Inconsistent parameter order. Every selection becomes several URLs.
  • Canonicals that lie. A filter page with different products canonicalised to the category; the engine ignores the hint and indexes both.
  • Blocking what is linked everywhere. Indexed without content, and no signal flow.
  • Sort and view parameters in the sitemap. The sitemap becomes the problem.
  • Forgetting internal search. ?q= URLs are faceted navigation by another name.

The site architecture guide covers where tier-one filter pages sit in the hierarchy; the e-commerce hub places this decision among the others a store makes.

Frequently asked questions

Should filter pages be blocked in robots.txt?

Only for combinations that should never be crawled and are not linked from anywhere that matters. Google's guidance notes that blocking prevents crawling but not indexing if the URL is linked, and that blocked URLs cannot pass signals through canonicals. The usual pattern is canonical or noindex for filters the crawler will find, and robots rules for deep combinations and sorts.

Is it better to use parameters or paths for filters?

Google's guidance treats both as workable if the structure is consistent and the engine can tell views apart. Paths suit the small set of filters chosen for indexing, because they read as pages and carry their own content; parameters suit the rest, because they are easy to recognise and manage as a group. Mixed usage, with the same filter expressed both ways, is what causes trouble.

What about sort orders and view options?

They never deserve indexing: the same products in a different order is the definition of a duplicate. Canonicalise sorted and paged-size views to the default view, or implement them without changing the URL.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Part of these learning paths

Sources

  1. Crawling and managing faceted navigation — Google Search Central
  2. Large site owner's guide to managing your crawl budget — Google Search Central

Related