What is a sitemap and Why Use a Sitemap for website?

What is a sitemap and Why Use a Sitemap for website?

What is a sitemap?

 A sitemap is a file where you can list the web pages of your site to tell Google and other search engines about the organization of your site content. Search engine web crawlers like Googlebot read this file to more intelligently crawl your site.

Also, your sitemap can provide valuable metadata associated with the pages you list in that sitemap: Metadata is information about a webpage, such as when the page was last updated, how often the page is changed, and the importance of the page relative to other URLs in the site.

If your site’s pages are properly linked, our web crawlers can usually discover most of your site. Even so, a sitemap can improve the crawling of your site, particularly if your site meets one of the following criteria:

Your site is really large. As a result, it’s more likely Google web crawlers might overlook crawling some of your new or recently updated pages.

Your site has a large archive of content pages that are isolated or well not linked to each other. If you site pages do not naturally reference each other, you can list them in a sitemap to ensure that Google does not overlook some of your pages.

Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.

Your site uses rich media content, is shown in Google News, or uses other sitemaps-compatible annotations. Google can take additional information from sitemaps into account for search, where appropriate.

Why Use a Sitemap

Using sitemaps has many benefits, not only easier navigation and better visibility by search engines. Sitemaps offer the opportunity to inform search engines immediately about any changes on your site. Of course, you cannot expect that search engines will rush right away to index your changed pages but certainly the changes will be indexed faster, compared to when you don't have a sitemap.

Also, when you have a sitemap and submit it to the search engines, you rely less on external links that will bring search engines to your site. Sitemaps can even help with messy internal links - for instance if you by accident have broken internal links or orphaned pages that cannot be reached in other way (though there is no doubt that it is much better to fix your errors than rely on a sitemap).

If your site is new, or if you have a significant number of new (or recently updated pages), then using a sitemap can be vital to your success. Although you can still go without a sitemap, it is likely that soon sitemaps will become the standard way of submitting a site to search engines. Though it is certain that spiders will continue to index the Web and sitemaps will not make the standard crawling procedures obsolete, it is logical to say that the importance of sitemaps will continue to increase.

Sitemaps also help in classifying your site content, though search engines are by no means obliged to classify a page as belonging to a particular category or as matching a particular keyword only because you have told them so.

Having in mind that the sitemap programs of major search engines (and especially Google) are still in beta, using a sitemap might not generate huge advantages right away but as search engines improve their sitemap indexing algorithms, it is expected that more and more sites will be indexed fast via sitemaps.

Generating and Submitting the Sitemap

The steps you need to perform in order to have a sitemap for your site are simple. First, you need to generate it, then you upload it to your site, and finally you notify Google about it.

Depending on your technical skills, there are two ways to generate a sitemap - to download and install a sitemap generator or to use an online sitemap generation tool. The first is more difficult but you have more control over the output. You can download the Google sitemap generator from here. After you download the package, follow the installation and configuration instructions in it. This generator is a Python script, so your Web server must have Python 2.2 or later installed, in order to run it.

The second way to generate a sitemap is easier. There are many free online tools that can do the job for you. For instance, have a look at this collection of Third-party Sitemap tools. Although Google says explicitly that it has neither tested, nor verified them, this list will be useful because it includes links to online generators, downloadable sitemap generators, sitemap plugins for popular content-management systems, etc., so you will be able to find exactly what you need.

After you have created the sitemap, you need to upload it to your site (if it is not already there) and notify Google about its existence. Notifying Google includes adding the site to your Google Sitemaps account, so if you do not have an account with Google, it is high time to open one. Another detail that is useful to know in advance is that in order to add the sitemap to your account, you need to verify that you are the legitimate owner of the site.

Currently Yahoo! and MSN do not support sitemaps, or at least not in the XML format, used by Google. Yahoo! allows webmasters to submit “a text file with a list of URLs” (which can actually be a stripped-down version of a site map), while MSN does not offer even that but there are rumors that it is indexing sitemaps when they are available onsite. Most likely this situation will change in the near future and both Yahoo! and MSN will catch with Google because user-submitted site maps are just a too powerful SEO tool and cannot be ignored.

You can learn how to create indices and more about sitemaps at sitemaps.org.

After you’ve created your sitemaps (and potentially sitemap indices), you’ll need to register them with the various search engines. Both Google and Bing encourage webmasters to register sitemaps and RSS feeds through Google Webmaster Tools and Bing Webmaster Tools.

Taking this step helps the search engines identify where your sitemap is — meaning that as soon as the sitemap is updated, the search engines can react faster to index the new content. Also, content curators or syndicators may be using your RSS feeds to automatically pull your content into their sites.

Registering your sitemap (or RSS feed) with Google and Bing gives the search engines a signal that your content has been created or updated before they find it on the other sites. It’s really a very simple process with both engines. 

To submit a sitemap to Google:

1. Ensure that the XML Sitemap is on your web server and accessible via its URL.
2. Log in to Google Webmaster Tools.
3. Under “Crawl,” choose “Sitemaps.”
4. Click on the red button in the upper right marked “Add/Test Sitemap.” Enter the URL of the sitemap and click “Submit Sitemap.”

To register a sitemap with Bing:

1. Ensure that the XML Sitemap is on your web server and accessible via its URL.
2. Log in to Bing Webmaster Tools.
3. Click on “Configure My Site” and “Sitemaps.”
4. Enter the full URL of the sitemap in the “Submit a Sitemap” text box.
5. Click “Submit.”

Another great reason to register sitemaps with Google specifically is to catch Sitemap errors. Google Webmaster Tools provides great information about the status of each Sitemap and any errors it finds: