An XML sitemap is a structured text file that lists the addresses of a website’s pages, along with metadata that describes their content and relative priority. The format is defined by the XML standard and follows a shared schema, so that search engines can read and interpret it without ambiguity. Its main job is to help crawlers discover a site’s URLs faster and more thoroughly than by following hyperlinks alone.
The protocol was introduced in 2005 by Google, Yahoo! and Microsoft as a joint initiative, later consolidated into an open standard published on sitemaps.org. Since then it has been adopted by the major search engines and is a well-established tool in the kit of anyone working on search engine optimization. It does not guarantee that pages get indexed, but it makes the process considerably easier.
What an XML sitemap is
The file has a .xml extension and contains a list of url elements, each referring to a single page. Every element includes at least the full address of the resource, expressed as an absolute URL. Optional fields can be added, such as the date of the last modification, the estimated update frequency and a priority relative to the other pages of the site.
The format is designed to be readable by both machines and humans, but its primary purpose is communication with crawlers. A well-built file reduces the risk that important pages stay out of the index, especially when they can only be reached through complex or hard-to-find navigation paths.
How it differs from other formats
There are also sitemaps in plain text format and in RSS or Atom format, but the XML version is the most widespread and the richest in information. The text format simply lists the addresses, while the RSS one is meant for frequently updated content, such as blogs and news sites.
What information it can contain
An XML sitemap has four main elements. The first is the page address, which is mandatory. The second is the date of the last modification, which helps search engines understand whether the content has changed since the last crawl. The third is the update frequency, an indicative value that suggests how often the content changes. The fourth is priority, a number between zero and one that indicates the relative importance of the page within the site.
These fields are not binding for search engines, which treat them as hints. Even so, filling them in sensibly helps communicate the site’s structure and editorial intentions more precisely. A confusing or oversized sitemap, on the other hand, can be less useful than a simple list of addresses.
Technical limits to respect
A single sitemap cannot contain more than 50,000 URLs and cannot exceed 50 megabytes in uncompressed size. For very large sites you use a sitemap index, that is, a file that lists several separate sitemaps. The file must be encoded in UTF-8 and the addresses must belong to the same domain as the file itself.
How to create and publish it
You can create it manually by writing the XML file, or automatically through content management systems that generate the sitemap dynamically. There are also dedicated tools that analyze a site and produce the file from the pages they find. In any case, it is good practice to check that the listed addresses are actually reachable and return a positive response status.
Publishing means placing the file in an accessible location, typically in the site’s root directory. The traditional path is /sitemap.xml, but any name and location will do, as long as the address is communicated to search engines. You communicate it through the robots.txt file or through the webmaster administration tools provided by each search engine.
Updating and maintenance
A sitemap is not a static document: it needs updating when the site’s pages change. If the file does not reflect the real state of the URLs, it loses much of its usefulness and can even confuse crawlers. On dynamic sites, automatic generation largely solves the problem, but a periodic check is still worthwhile.
| Aspect | XML sitemap | Text sitemap | RSS/Atom sitemap |
|---|---|---|---|
| Available metadata | Complete | None | Partial |
| Suitable for large sites | Yes, with an index | Yes | No |
| Typical use | Structured sites | Simple lists | Blogs and news sites |
| Search engine support | Widespread | Widespread | Limited |
| Priority and frequency | Supported | Not supported | Not supported |
Benefits and limits in optimization
The main benefit is the speed with which search engines can discover new or updated pages. This is especially useful for sites with a lot of content, with sections that are poorly linked to each other, or with pages reachable only through deep paths. A well-made sitemap also reduces the load on servers, because it directs crawlers toward the relevant URLs.
The main limit is that being in a sitemap does not automatically mean being indexed. Search engines still assess the quality and usefulness of the content, and may decide not to include a page even if it is listed. Moreover, a badly configured sitemap or one that is out of sync with the site can backfire and waste the available crawl budget.
Common mistakes to avoid
Among the most frequent mistakes are including non-canonical URLs, listing addresses that return errors, using untruthful dates and duplicating entries. Another mistake is declaring high priority for every page, which effectively cancels out the field’s effect. Finally, avoid including pages blocked by the robots.txt file, because the message would be contradictory.
Steps to build a correct sitemap
- List all the canonical URLs of the site, excluding duplicates and unnecessary parameters.
- Check that every address responds with a positive status and does not redirect elsewhere.
- Assign each page a realistic and verifiable last modification date.
- Indicate an update frequency consistent with the nature of the content.
- Assign differentiated priority values, reserving the highest for key pages.
- Split into multiple files if you exceed the size or URL count limits.
- Publish the file and report its address in the robots.txt file or in the webmaster tools.
- Check the sitemap’s status periodically and update it as the site changes.
Frequently asked questions
Is an XML sitemap mandatory?
No, it is not mandatory. Search engines can discover pages by following internal and external links too, but a sitemap makes the process faster and more complete, especially on large or complex sites.
Does it help you rank better in the results?
It does not directly affect ranking. Its role is to make it easier to discover and crawl pages; position in the results depends on other factors, such as content quality and the site’s authority.
Where should the file be placed?
Traditionally in the site’s root directory, with the name sitemap.xml, but any accessible location will do. What matters is that the address is communicated to search engines through robots.txt or the dedicated tools.
How many URLs can a single file contain?
The limit is 50,000 URLs per file and 50 megabytes in uncompressed size. Beyond these thresholds, you need to split the content into several sitemaps linked by an index.
Does it need to be updated manually?
It depends on the content management system. Many platforms generate and update the sitemap automatically, while on static sites maintenance may require periodic manual work.
Can it be compressed?
Yes, the file can be compressed in gzip format to reduce its size. In that case the address you communicate to search engines includes the extension of the compressed file.
Related entries:
Creating an XML sitemap SEO sitemap