Site Maps - What they are and why you need them
As with all the major search engines, a Google Sitemap (or Google site map) is a useful tool to assist in navigating a website. There are two main types of Sitemaps, each serving its own purpose. Sitemaps designed for search engine optimization (SEO) assist web crawlers in indexing your website, while Sitemaps designed for your site's visitors assist them in navigating your content.
Sitemaps for SEO
Search engine web crawlers (also referred to as spiders, robots, bots, and, in the case of Google, Googlebots) roam throughout the Internet to see what's there. As these crawlers visit each website, they incorporate the content into the search engine results pages (SERPs). This allows Internet users to utilize search engines such as Google, Yahoo!, Ask, and Bing to perform a web search and discover the relevant sites.
An XML Sitemap file enables you to direct these web crawlers to all the pages on your website that you want indexed. It can be used alone or as a complement to a robots.txt file (used to tell the web crawlers which pages should not be indexed).
As an alternative, some search engines (including Google) allow you to submit your RSS or Atom 1.0 feed instead of an XML file. If you operate a blog, this can be a viable solution. Many blog platforms such as WordPress can generate and submit the feed for you. Text files containing only page URLs (no images or videos) may also be acceptable as simple Sitemaps.
There is no requirement to have an XML Sitemap (or an alternative) on your website, but it can certainly help your content be found by search engines. It can also assist your site in gaining a favorable position in the SERPs.
Sitemaps for Visitor Navigation
Whereas the SEO-oriented Sitemap mainly targets web crawlers, an HTML Sitemap is meant for the human visitors to your site. Essentially, it is just like any other page on your website and can be built on the same template. It is intended to be viewed by visitors and used to find the desired content.
Visitors often use a Sitemap if they encounter difficulty finding what they are seeking on a website. Therefore, your HTML Sitemap should be simple and easy to understand. Avoid any unnecessary images, videos, or other clutter. It may be useful, however, to include a short description for each link.
The more pages you have, the more important it is to arrange your pages in relevant groupings and to identify any practical structure or hierarchy among the pages. If your site contains so many pages that an exhaustive HTML Sitemap is impractical, determine the most important pages and link to those URLs from your Sitemap. You can also create categories with links to other pages containing sub-menus.
Typically, a web designer will place a link to the HTML Sitemap in the universal footer of the website. Accordingly, this is where most visitors will look to find that link on your site. In addition to the footer, other recommended locations to place the link include your “Help” page and your custom 404 “not found” page.
Whenever you add or delete pages on your site, remember to update your visitor Sitemap. Otherwise, your Sitemap can quickly become outdated and of limited use for visitors.
Google Webmaster Tools (as well as the counterparts of other search engines) can assist you in creating and submitting your XML Sitemap. Because the syntax of the XML file must be precise, it is vital to use the correct protocol. If you prefer, you can take advantage of a reputable third-party Sitemap generator that can do the job for you.
Webmaster Tools also provides information about Sitemap variations, including Video, Mobile, News, and Geo Sitemaps. Depending on your needs and the nature of your website, this information can greatly impact your level of success.
In addition, Webmaster Tools provides you with instructions for submitting your Sitemap. With Google, this can be done easily through an online submission form or by using your robots.txt file. Simply include the following line in the robots.txt file, without the quotation marks and edited with your relevant information:
Once you submit your Sitemap, remember to keep it current. You can use Webmaster Tools to submit updated Sitemaps as often as needed.
Should your site have Sitemaps for web crawlers and visitors? It is definitely worth considering. There are no penalties for having them, and there are disadvantages of not having them. Granted, if your website is relatively small and you can include all the necessary links within your main menu, you can get away without having an HTML Sitemap for visitors. Almost every website, though, regardless of its size, can benefit from having an XML Sitemap that is properly coded, up-to-date, and submitted to the dominant search engines.