XML Sitemap Checklist: Limits, Rules and Common Mistakes

A sitemap is a file that lists the URLs you want search engines to know about. It does not force indexing, but a clean one speeds up discovery, and a messy one teaches crawlers to ignore it. Use this checklist before you submit a sitemap in Search Console or send URLs in bulk.

The rules that are fixed

One sitemap file holds at most 50,000 URLs and 50 MB uncompressed. Above that, split it and list the parts in a sitemap index file.
Use absolute URLs with the exact protocol and host you want indexed (for example https and the www or non-www version you chose).
The sitemap must be on the same site it describes, and reachable without a login.
Reference it from robots.txt with a Sitemap: line so any crawler can find it.

Content checklist

Check	Why it matters
Every URL returns 200	Redirects, 404s and 5xx errors waste crawl attention and lower trust in the file.
No noindex URLs	A sitemap says 'index this'; a noindex tag says the opposite. Search Console reports the conflict.
No URLs blocked by robots.txt	Google cannot fetch them, so they sit as 'Blocked by robots.txt'.
Each URL is the canonical version	Listing parameters, tracking variants or duplicates splits signals. List only the preferred URL.
lastmod is accurate	Update it only when the page content really changed. Google ignores lastmod once it learns a site always sets it to 'now'.
priority and changefreq	Google ignores both. You can leave them out.
No duplicate entries	The same URL listed twice, or with and without a trailing slash, is noise.
Pages are worth indexing	Do not list internal search results, cart pages or thin tag archives.
No overlapping sitemaps	If a parent site and a subdomain both list the same URLs, engines see duplicate submissions. One host should own each URL.

Common mistakes we see

A stale lastmod on every URL, often set once at build time. It looks like a static site that never changes, even if you publish weekly. Regenerate it, or set it from the file's real modification date.
Mixed hostnames: the sitemap lists http URLs or the wrong www variant while the pages canonicalise to something else.
Listing everything that the CMS can generate, including paginated archives and filter pages.
Sitemap blocked by robots.txt or served with an HTML error page and a 200 status, so crawlers receive HTML instead of XML.
Never resubmitted after a redesign changed URL structure, leaving hundreds of redirects in the file.

How to test it

Open the sitemap URL in a browser. You should see raw XML, not a web page.
Spot-check a sample of URLs for status code, canonical tag and robots meta tag. For a few hundred URLs, a crawler or a script is easier than clicking.
Submit it in Search Console's Sitemaps report and check for "Couldn't fetch" or "Success" with a discovered-URL count that matches what you expect.
After a week, compare the number of submitted URLs with the number indexed. A large gap points you to the Page indexing report for the reasons.

Sitemap and submission tools work together

Submitting URLs through an API is a way to nudge discovery of specific pages. The sitemap is the standing list that tells search engines your whole structure. If the two disagree, for example you submit a URL that the sitemap omits or marks as noindex, you are sending mixed signals. GrowLytics' URL Indexer can load URLs from a sitemap and filter out ones that are malformed or off-domain before you spend credits.

References: Google: build and submit a sitemap, sitemaps overview, robots.txt guide.

Frequently asked questions

No. It helps discovery. Whether a page is indexed still depends on its quality and on Google's decision.

No. Google ignores both fields. Leaving them out keeps the file simpler.

Only when the page's real content changes. Setting it to the current time on every build teaches search engines to disregard it.

Yes, and large sites should. Use a sitemap index file to list them, and keep each under 50,000 URLs and 50 MB.

XML sitemap checklist: what to include, what to leave out