Definition
Google's documentation describes noindex as a rule that blocks a page from appearing in search, and gives two ways to set it:
- A robots meta tag in the HTML head, naming either all crawlers or a specific one, with the value
noindex. This suits ordinary web pages. - An X-Robots-Tag response header sent by the server with the same value. This suits files that have no HTML head, such as PDFs and images, and rules applied across whole sections at server level.
The directive acts at the indexing stage. The page can still be crawled, its content read and, for a time, its links followed. Google documents that a page kept on noindex for a long time is eventually crawled less often and that its links may stop being followed, so noindex is not a way to keep link signals flowing through a page that is excluded. A noindex rule placed inside robots.txt is not supported.
In practice
noindex belongs on pages that must exist for visitors but add nothing as search results:
- Thin utility pages such as thank-you and confirmation pages, account pages and print views.
- Internal search results, which create an endless supply of low-value URLs.
- Filter combinations on stores that are crawlable through links but should not rank. The faceted navigation guide uses noindex for filter views too different from any page to canonicalise honestly, and the canonical tag for the rest.
- Staging and preview sites, where it is a safety net, though password protection is the stronger control.
noindex, robots.txt and canonical
| Control | Acts on | Use it for | Do not use it for |
|---|---|---|---|
| noindex | Indexing | Keeping a crawlable page out of results | Saving crawl on huge URL spaces |
| robots.txt | Crawling | Keeping crawlers out of infinite spaces and private paths | Removing a page from results |
| Canonical tag | Choice between duplicates | Consolidating signals onto a preferred URL | Excluding a page that has no equivalent |
The three are often confused because they all affect what appears in search. The crawling and indexing guide sets out the sequence: robots rules decide what is fetched, noindex decides what is kept, and the canonical tag is a hint about which duplicate represents a group. Combining noindex with a canonical pointing elsewhere sends contradictory instructions, and blocking a noindexed page in robots.txt hides the instruction, so a URL that other sites link to can stay listed without a description.
Leftover noindex after launches
The most expensive noindex is one nobody meant to keep. Staging sites are rightly set to noindex; when the template or a site-wide setting is copied to production, the live site tells search engines to drop every page. The web page publishing SOP checks each live page for exactly this, and the SEO page publishing checklist lists a noindex left over from staging among the settings to confirm before a page goes live. Language versions need the same care: the hreflang guide notes that a noindexed version breaks the group of alternates it belongs to.
Search Console reports excluded pages with the reason, including exclusion by noindex. The Search Console error triage SOP separates deliberate exclusions, which are working as intended, from pages that should be indexed and are not.
A staging setting that went live
Ulverby Paint Supplies launches a redesigned site on a Tuesday. The new template still carries the robots meta tag the agency used on staging. By the following Monday organic visits have fallen from about 1,900 a day to 620, and Search Console's page indexing report shows 2,300 pages excluded by the noindex tag, a figure rising daily. The tag is removed from the template that afternoon, the sitemap is resubmitted and the most important category pages are inspected and submitted for indexing. Organic visits return to 1,800 a day within three weeks, and a template check is added to the launch plan.
Related terms
The Canonical TagTells search engines the preferred URL. is the alternative when a page has a genuine equivalent; noindex is for pages that should simply not appear. An XML SitemapA file listing the URLs a site wants search engines to discover and crawl. should never list a noindexed URL, since the two would contradict each other. GSCGoogle's search performance and indexing tool. is where noindex exclusions are reported and where an accidental one is usually first noticed. Because a noindexed page is still fetched, noindex does nothing for Crawl BudgetHow much a search engine crawls a site.; robots rules are the tool for that.
Common confusions
- Blocking in robots.txt is not noindex. A blocked page cannot be read, so its noindex never takes effect.
- noindex is not instant. The page drops out only after it is recrawled.
- noindex is not a link-flow tool. Long-standing noindex pages may eventually have their links ignored.
- noindex and canonical do not combine. One says index nothing, the other says index that page instead.
- Excluded by noindex is not always an error. It is often the intended state.
