Most of the time as SEO’s, our job is to get more visibility for the websites that we work on. We get them indexed and ideally, get them moved up in the rankings for keywords that we want to rank for. But there are times when a website needs to go away, and a client wants it completely removed from Google search results.  In many cases simply deleting the content from your website, or even letting your domain expire works. But sometimes Google is stubborn, and does not want to remove your old website. This was the case for us recently, and we had to “force” Google to remove it.

Case website stats:

  • the website had about 200 pages
  • designed WordPress
  • about 1 year old
  • we have full control of the hosting environment
  • full access to Google Search Console (verified)
  • the website needed to be “removed” because it was no longer in use

In the past, getting a website to disappear from Google search results was rather easy. I would just delete the entire website and serve out a 404. Google would crawl and the results would slowly disappear.

Don’t use site: to test index status anymore

There are several schools of thought when trying to remove a domain from Google. I’ve done this a bunch of times recently with varying success. It wasn’t until my friend Bill Sebald spoke to John Mueller (Google) asking about the site: operator and still-indexed sites. John dropped this nugget of information which apparently has been known for some time:

So if you are an old-school SEO like me who is used to using site: to check the index status of a domain, you should no longer rely on it. Instead, you need to fire up Google Search Console and check that way. Annoying, I know.

For me, this brings up a danger: what if a client (legally) needs the website removed even from Google’s site: “cache” and all of its title / meta information? Example: your website was indexed in Google, and within the meta tags displayed something proprietary or illegal. Since it seems the site: operator will show a “shadow” of the old site even when the site is gone, we need a new solution. If I were in this situation, I would rebuild the site with jibberish, dummy text, lorem ipsum, or irrelevant information. Get the new version of the site indexed, then remove that version.

For me this was a huge deal. Not being able to trust the site operator for debugging purposes really throws a wrench into my workflow. It still “works” for the most part, just not for checking removals.

Removing a site from Google

This is the method that we used after speaking to a number of other SEO consultants. We started by deleting all files from the host (make sure you have a backup) and starting with a blank slate. From there we created 4 files:

  • an .htaccess
  • a robots.txt
  • an index.html
  • and a sitemap.xml

The .htaccess

This custom .htaccess file is short and sweet. It allows the robots.txt, sitemap.xml, and Google Search Console’s verification file. It then tells any browser that encounters this domain that it is 410, gone.

.htaccess
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/robots\.txt$
RewriteCond %{REQUEST_URI} !^/sitemap\.xml$
RewriteCond %{REQUEST_URI} !^/google010dec22d6xxxxxx\.html$
RewriteRule ^(.*)$ - [G]

If done properly, browsers will see this:

a robots.txt

I don’t think a robots.txt file is really needed here, but just to be safe we wanted to allow all bots free reign on the site. Remember, even though you want the site removed from Google you still need to allow Googlebot to crawl the site. You can test your robots.txt on Google Search Console for safe keeping.

User-agent: *
Disallow:

an index.html file

Just a blank index.thml file, not really necessary just for testing purposes. Originally I thought about putting robots / noindex in this file, but really didn’t want to confuse Googlebot so just left it blank since the 410 would hit first.

  






The sitemap.xml file


Again, not totally necessary. Some SEO’s recommend rebuilding the old sitemap.xml file from the original site and submitting to Google Search Console. I chose not to, and had great results.





https://example.com/
2019-12-20T10:46:35+00:00
daily
1.00





An extra step: allow me to put on my tinfoil hat for a second. If you really want to send Google a signal that your site has changed, point your domain to a new host (and new IP) then stick your robots, sitemap, .htaccess, and index.html on that server. This is just a theory that it may help Googlebot see a drastic change in your site’s infrastructure, it can’t hurt you.


Remember, the above steps are to remove your website from Google Search Results and does not apply if you are moving your website. If you are moving your website(domain) there are a completely different set of steps to take.


Also note that if you take the above steps verbatim, you will null and void any kind of 301 redirects you might have in your .htaccess file.


Any questions, please feel free to hit me up on Twitter or if you need help with your website you can always contact us.


Thank you!