XML and Website Sitemap
Think of your XML website sitemap as a suggested driving directions for when Google crawls your website. Google is going to crawl your website regardless of if you have a sitemap or not, but having one will help optimize Googlebot’s time on your website. You can also identify which pages are the biggest priority, and which ones change the most. For instance if you have a “jobs” page that updates frequently, you would want Google to crawl that page much more often than you would your “about us” page which probably never changes.
If you are a WordPress user there are countless plugins that can create an XML sitemap for you. Same goes for other frameworks such as Joomla, Prestashop, etc. Very rarely will you have to actually create a sitemap on your own, but if you do there are a number of tools out there that will do this for you.
But what if you don’t have a CMS such as WordPress or modern day web framework. Well, there are a few options. The first one is to build it manually and write the XML by hand. You’d do this by making a list of all of your pages from within your website, and following this format:
If you don’t have the patience for this, you can use a program such as Screaming Frog SEO Spider to scrape your entire website and generate the sitemap for you. It is basically a 4 step process:
- scrape your website using Screaming Frog
- export your XML sitemap to your computer
- verify your XML sitemap file for errors
- upload your sitemap to your server
Letting Google Know About Your XML Sitemap
Letting Google Know About Your XML Sitemap
Once you’ve got an XML sitemap generated and uploaded to your server, it is generally a good idea to inform Google about this. Sure, they can probably find out on their own but this is recommended. The first way to let Google know about your XML sitemap is to add your sitemap to Google Search Console (formerly Google Webmaster Tools). This is generally a 1 step process and only takes a moment if you are logged in. Once your sitemap has been added you can see all kinds of nifty stats about your website:
Another way to let Google know about your sitemap is by adding it to your robots.txt file. This is generally one line of code that looks like this:
If you have multiple sitemaps you can just repeat that line as many times as necessary, changing the values of course.
XML Image and Video Sitemaps
If you’ve gotten the concept of a regular XML sitemap that you’ll have no problems understanding XML image and video sitemaps. These sitemaps are created for websites that have a lot of videos and images that typically have complex hierarchies. For instance a large website such as Vimeo that is almost completely made up of videos should have a video sitemap.
Having a specific type of sitemap also tells Google what your content is exactly. Having a video sitemap for instance will help Google give you a video snippet within the search results, but no guarantees ever.
Google has recommended in the past not only creating an XML sitemap for search engine crawlers, but to create a website map for users as well. We’ll talk a little bit more about that in our next section.
To clear things up, yes there is a difference between a website sitemap and an XML sitemap. As an example here is the difference between our XML sitemap and our website or user sitemap. The XML sitemap is meant for bots, the website or user sitemap is meant for people.
The user sitemap is useful for a number of reasons. For starters, it gives your visitors a map in case they aren’t sure where they are or what they are looking for. From an SEO perspective that is helpful for a number of reasons. It not only creates a perfect internal link structure, but it helps keep visitors on your site longer as well.
Google recommends a user sitemap for users that are having difficulty locating pages within your website, but a sitemap shouldn’t always be an afterthought. We recommend that people looking to develop or design a new website considers designing their sitemap in a program such as Excel or Visio in the beginning, especially for larger websites. This way you can get a visual representation of how your website hierarchy looks instead of just making a good guess.
The New York Times has an absolutely exquisite website sitemap. With a website this big, they can’t have every page ever written linked within the page, so they provide a basic “site map” for their users. Since they have content daying back to 1851, they are in a bit of a different category than most of the web.
Sites smaller than the NYT but larger than most like Apple makes their website sitemap a bit different. Instead of listing every link from within their website, they link to every section of the website or every category.
You may not think that sitemaps get visited by most users, but give it a try for a few months and see for yourself. On most websites I manage I see quite a lot of traffic going to these pages, especially ones with complex navigation.