Why parameters multiply URLs
A category template that accepts a sort order, a page size, three filters and a tracking tag does not serve one URL; it serves every combination of those values, in every order a link might write them. None of that is visible on the page, which looks identical whichever address loaded it. To a crawler, each distinct address is a distinct URL to fetch, compare and either index or fold into another.
Google's page on consolidating duplicate URLs counts the results of sorting and filtering among the ordinary causes of duplicate content, and its faceted navigation documentation describes what happens when such URLs are left uncontrolled: crawlers spend their effort on variations that add nothing, and new or updated pages are discovered more slowly. Its large-site guide to Crawl BudgetHow much a search engine crawls a site. recommends consolidating duplicates and keeping URLs that should never be crawled out of the crawl, while stating that crawl limits are a concern mainly for large or fast-changing sites. On a small site, parameters are more often a duplication and reporting problem than a crawling one; on a large catalogue they are both.
Parameters are also the cheapest URLs to create by accident. A developer adds a ref value to a banner, a
plugin appends a session identifier, a marketing team puts UTM tagsURL tags identifying traffic source. on an internal
promotion, and each change produces a new family of addresses without anyone deciding that it should. The
audit below finds what already exists and sets the rule that catches what comes next.
Before you start
- A crawl with query strings kept. Configure the site crawler so it does not strip or ignore parameters, and let it follow links that carry them, within a sensible limit per template.
- Crawler requests from the server logs. A few weeks, filtered to search engine crawlers, as the log file analysis tutorial describes. Logs reveal parameters that no current link creates: old campaign tags, session identifiers from a previous platform, URLs other sites link to.
- Search Console access for the Page Indexing report, whose example URLs show which parameter addresses Google has found and what it did with them.
- The template list, mapping each URL pattern to its page type.
- Someone who knows the code. For every parameter you will need to know what reads it and why it was added; a developer or platform owner answers in minutes what testing takes hours to infer.
- The analytics owner, because tracking parameters belong to campaign reporting, and their naming is set by the convention in the UTM and naming conventions guide.
The steps
Collect parameters from a crawl, logs and Search Console
Extract every query-string key from the three sources into one list. For each key, record how many distinct URLs carry it, one example URL, the templates it appears on and the source that found it. Normalise case before counting, since
Sortandsortare one parameter to a person and two to a server. Add parameter-like path segments too, such as/sort/price/or/page/2/, because they behave the same way. Then check the places crawls miss: the XML sitemap, internal search forms, script-generated links and the URLs in Search Console's examples for duplicate and alternate statuses.Record what each one does to the page
Load a representative URL with and without each parameter and compare what changes. Does the main content change, does the set of items shrink, does only the order or layout change, or does nothing visible change at all? Ask the developer what the code does with it, which sometimes differs from what the page shows: a parameter can set a cookie, switch a currency or select a variant behind the scenes. Note the values each parameter takes, whether an empty or nonsense value returns an error or a normal-looking page, and whether any internal link on the site carries it. Finding why a parameter was added before deciding its rule is an instance of Chesterton's fence, since a parameter with no visible effect may still feed a report or a cookie.
Classify each parameter
Give every parameter one class from the list below. Classes, not individual parameters, get rules, which is what keeps the register short enough to maintain.
Class What it does Typical examples Changes content Selects a different document id,article,langon a site without language pathsNarrows Shows a subset of a list colour,brand,priceReorders Same items, different order or layout sort,view,per_pagePaginates Splits a list into numbered pages page,pTracks Records where a visit came from utm_source, click identifiers,refHolds a session Carries state for one visitor sessionid,sid,basketSearches Runs the site's own search q,s,searchA parameter that fits two classes is usually two parameters sharing a name, which is worth fixing in code.
Set the rule for each class
For each class, decide four things: whether crawlable links should point to it, where its canonical points, whether robots rules block it and whether it appears in the sitemap. Content-changing parameters are pages: linked, self-canonical and in the sitemap. Narrowing parameters follow the tiers in the faceted navigation guide, which decides which filter combinations deserve pages. Reordering parameters canonicalise to the default view and are not linked as crawlable anchors. Paginated URLs stay crawlable and self-canonical, as the site architecture guide explains. Tracking parameters are never used on internal links and canonicalise to the clean URL. Session parameters should not be in URLs at all. Search results are kept out of the index and the crawl.
Choose the mechanism for each rule
Prefer the mechanism that stops the problem earliest. Not creating the URL is best: a sort control that updates the page without changing the address, a session held in a cookie. Next is a redirect, which suits normalisation, such as forcing one parameter order or removing a session identifier. Google's page on duplicate URLs describes redirects and canonical tags as strong signals and sitemap inclusion as a weak one, and advises against using robots rules or noindex to choose a canonical. A Canonical TagTells search engines the preferred URL. suits variants that must stay reachable. A robots.txtA file at a site's root telling crawlers which URLs they may and may not fetch. rule suits patterns that should never be fetched, with the caveat from the large-site guide that blocked URLs can stay in the crawl queue and that a crawler cannot read a canonical on a page it may not fetch. When the choice is between a canonical and a redirect, the comparison of canonical tags and 301 redirects sets out the test.
Clean internal links and sitemaps
Rules mean little while the site keeps linking to the URLs they are meant to suppress. Google's link guidance explains that it follows links written as anchor elements with an
href, so every Internal LinkA link from one page to another page on the same website. that carries a reordering, tracking or session parameter is an invitation to crawl it. Crawl again and list internal links by parameter class. Fix them at the template: navigation, breadcrumbs, pagination links that carry the current sort, related-item widgets, promotional banners with campaign tags. Sort and view controls that need no URL can be buttons rather than anchors. Regenerate the sitemap so it lists only clean, canonical addresses.Write the register and the rule for new parameters
The register is one row per parameter: name, class, values, what it does, owner, rule, mechanism, the date it was reviewed and any exception. Store it where developers and marketers both look, next to the URL rules in the developer handbook. Then write the rule for parameters that do not exist yet: no new parameter ships without a row; until it is classified, it is treated like a tracking parameter, never linked internally and canonicalised to the clean URL; and any parameter meant to create indexable pages needs the owner of SEO to agree first. The SEO requirements tutorial shows how to put that rule into the specification for a new template.
Verify in a recrawl, the logs and the indexing reports
Recrawl and confirm that internal links now reach only permitted classes, and that each sampled parameter URL returns the canonical, directive or redirect its rule requires; the SEO page checker reads those on a single URL. In the logs, compare crawler requests to each parameter class over a window of the same length as before the change. In the Page Indexing report, parameter URLs should move into "Alternate page with proper canonical tag" or out of the report, and fewer should appear under "Duplicate without user-selected canonical". Changes show gradually as URLs are recrawled.
A parameter register (illustrative)
Tinsprocket Homewares writes down its parameters
Tinsprocket Homewares, an invented online homewares store, runs the audit on its 120 category pages. The
crawl, the logs and Search Console together surface eleven parameters. A typical category was reachable at
about 40 distinct addresses: four sort orders, two page sizes, colour and material filters, an old sid
session parameter still present in links on archived blog posts, and utm_ tags that the marketing team had
added to homepage banners. The register the team wrote, in shortened form:
| Parameter | Class | Rule | Mechanism |
|---|---|---|---|
sort, view, per_page | Reorders | Not linked; canonical to default view | Controls changed to buttons; canonical tag |
colour, material | Narrows | Two colour pages per category built as real pages; rest canonical to category | Faceted rules from the store's filter tiers |
page | Paginates | Crawlable, self-canonical, linked | Plain links between pages |
utm_*, ref | Tracks | Never on internal links; canonical to clean URL | Banner links rebuilt without tags |
sid | Holds a session | Removed from URLs | Session moved to a cookie; old URLs redirected |
q | Searches | Noindex; not crawled | Noindex on template; robots rule for the pattern |
Eight weeks later, a recrawl finds an average of 4 addresses per category: the category, its pagination and its two colour pages. Crawler requests to parameter URLs in the logs fall from 61% of the total to 14%, and most remaining requests go to tagged URLs that other sites link to, which carry the canonical as intended.
Keeping the register current
Parameters arrive with features, campaigns, plugins and platform upgrades, so a register checked once drifts within months. Three habits keep it accurate. First, make the register part of release review: any change that adds or reads a query parameter names its row, and a change without one is returned. Second, compare the parameters in the logs with the register on a fixed schedule, quarterly on most sites and monthly on large catalogues; a key the register does not know is either a new feature nobody declared or an external source of links worth understanding. Third, review the narrowing class with the keyword map at the same time, because demand changes, and a filter that did not deserve a page last year may deserve one now.
The register also answers a question that comes up in every audit and migration: why a given URL behaves the way it does. When the duplicate without user-selected canonical diagnosis points at query strings, the register says whether the parameter is meant to exist, what its canonical should be and who owns the fix. The academy exercise on registering a job board's URL parameters from its crawl logs practises listing every filter, sort and tracking parameter with what it does and a rule for crawling, indexing and links.
Common mistakes
- Blocking tracking parameters in robots.txt. External links still point at them, and the crawler can no longer read the canonical that would consolidate them.
- Tagging internal links with campaign parameters. It creates duplicate URLs and, as the analytics guide explains, restarts the visit's traffic source.
- Rules per parameter instead of per class. A register of forty individual decisions is not maintained; seven classes are.
- Using noindex to choose a canonical. Google's duplicate URL page advises against it; noindex removes the variant instead of consolidating it.
- Fixing canonicals but not links. A canonical contradicted by every internal link is a weak signal.
- Ignoring parameter order and case.
?a=1&b=2and?b=2&a=1are two URLs unless the site normalises them. - No rule for new parameters. The next release recreates the problem the audit just fixed.
