Definition
The format is a plain XML file with one entry per URL. The required part is the address; the optional last-modified date tells a crawler which pages have changed since its last visit. Google's description of how search works places sitemaps in the discovery stage, next to following links from known pages: a sitemap tells the engine which URLs exist and which address the site prefers for each. The crawling and indexing guide adds the documented detail that Google uses the last-modified value when it is accurate and consistently maintained, and ignores it where it is not.
Large sites split their URLs across several sitemap files, because each file has a documented size limit, and list those files in a sitemap index. Most content management systems generate the files automatically, which is convenient and is also how unwanted URLs find their way in.
In practice
A sitemap works best as a clean statement of the site's canonical, indexable pages. In practice that means:
- Include every page the site wants to appear in search, at its canonical address, returning a normal response.
- Exclude redirected URLs, error pages, pages carrying a noindex directive, non-canonical duplicates, parameter and filter variants, internal search results and staging addresses.
- Keep it current. Add pages when they are published, remove them when they are retired, and update the last-modified date only when the content really changes.
On stores, the faceted navigation guide is explicit that the sitemap should declare only the indexable tier of filter pages, never the sort and view combinations, because a sitemap polluted with filter noise becomes part of the problem it was meant to solve.
Submission and monitoring
A sitemap reaches search engines in two ways: a line in the robots.txt file giving its address, and submission in Search Console. The Google Search Console tutorial covers submitting the sitemap or sitemap index and confirming it is read without errors, and explains why neither step makes pages rank: it makes the rest of the data interpretable, because the engine now knows which URLs the site considers canonical.
The useful reading is the gap between what the sitemap submits and what is indexed. A large gap means either the sitemap lists URLs that should not be there, or the listed pages are not meeting the engine's quality or crawl priority. The technical SEO audit guide compares the sitemap with a crawl of the site: pages in the sitemap that the crawl never reaches are orphans, and pages the crawl finds that the sitemap omits are either missing from it or should not exist.
Sitemaps in migrations and for hreflang
Two specialised uses come up often. During a move, the site migrations guide recommends submitting the new sitemap at launch and keeping the old one available for a period, so the old URLs are recrawled and their redirects discovered. For multilingual sites, the hreflang guide describes the sitemap as one of three places language alternates can be declared, and the one that scales best because a single generator can emit every version from the content database.
A sitemap trimmed to what the shop wants found
Wickstead Children's Books has a sitemap generated by its shop platform that lists 18,500 URLs: 6,200 live titles, 4,100 out-of-print titles that return a not-found page, and 8,200 filter pages for age, format and price. Search Console shows 5,400 of the submitted URLs indexed. The shop rebuilds the sitemap to list only canonical, in-stock title pages and its 40 curated category pages, and redirects out-of-print titles with a close equivalent to that book. Two months later the sitemap submits 6,240 URLs and 5,900 are indexed, and new titles appear in search within days rather than weeks.
Related terms
GSCGoogle's search performance and indexing tool. is where a sitemap is submitted and where its indexing gap is read. A page carrying noindexA robots directive telling search engines not to show a page in their results. should never be listed, because the sitemap would be asking for indexing that the page itself refuses. Every listed URL should be the one its Canonical TagTells search engines the preferred URL. names, so that the two signals agree. On very large sites, an accurate sitemap helps direct Crawl BudgetHow much a search engine crawls a site. toward pages that are new or changed.
Common confusions
- Listed is not indexed. A sitemap invites a crawl; quality and canonical decisions still apply.
- A sitemap does not replace internal links. A page reachable only through the sitemap gets none of the context or priority that links from related pages give.
- The HTML sitemap is something else. A page of links for visitors is navigation, not the XML file.
- More URLs are not better. Non-canonical and blocked URLs in the sitemap contradict the site's other signals.
- A fake last-modified date helps nothing. Dates that change without content changes teach the engine to ignore them.
