Case Study: (Intentionally) Removing a Website From Google Search

Most of the time as SEO’s, our job is to get more visibility for the websites that we work on. We get them indexed and ideally, get them moved up in the rankings for keywords that we want to rank for. But there are times when a website needs to go away, and a client wants it completely removed from Google search results.ย  In many cases simply deleting the content from your website, or even letting your domain expire works. But sometimes Google is stubborn, and does not want to remove your old website. This was the case for us recently, and we had to “force” Google to remove it.

Case website stats:

In the past, getting a website to disappear from Google search results was rather easy. I would just delete the entire website and serve out a 404. Google would crawl and the results would slowly disappear.

Don’t use site: to test index status anymore

There are several schools of thought when trying to remove a domain from Google. I’ve done this a bunch of times recently with varying success. It wasn’t until my friend Bill Sebald spoke to John Mueller (Google) asking about the site: operator and still-indexed sites. John dropped this nugget of information which apparently has been known for some time:

So if you are an old-school SEO like me who is used to using site: to check the index status of a domain, you should no longer rely on it. Instead, you need to fire up Google Search Console and check that way. Annoying, I know.

For me, this brings up a danger: what if a client (legally) needsย the website removed even from Google’s site: “cache” and all of its title / meta information? Example: your website was indexed in Google, and within the meta tags displayed something proprietary or illegal. Since it seems the site: operator will show a “shadow” of the old site even when the site is gone, we need a new solution. If I were in this situation, I would rebuild the site with jibberish, dummy text, lorem ipsum, or irrelevant information. Get the new version of the site indexed, then remove that version.

For me this was a huge deal. Not being able to trust the site operator for debugging purposes really throws a wrench into my workflow. It still “works” for the most part, just not for checking removals.

Removing a site from Google

This is the method that we used after speaking to a number of other SEO consultants. We started by deleting all files from the host (make sure you have a backup) and starting with a blank slate. From there we created 4 files:

The .htaccess

This custom .htaccess file is short and sweet. It allows the robots.txt, sitemap.xml, and Google Search Console’s verification file. It then tells any browser that encounters this domain that it is 410, gone.

.htaccess
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/robots\.txt$
RewriteCond %{REQUEST_URI} !^/sitemap\.xml$
RewriteCond %{REQUEST_URI} !^/google010dec22d6xxxxxx\.html$
RewriteRule ^(.*)$ - [G]

If done properly, browsers will see this:

a robots.txt

I don’t think a robots.txt file is really needed here, but just to be safe we wanted to allow all bots free reign on the site. Remember, even though you want the site removed from Google you still need to allow Googlebot to crawl the site. You can test your robots.txt on Google Search Console for safe keeping.

User-agent: *
Disallow:

an index.html file

Just a blank index.thml file, not really necessary just for testing purposes. Originally I thought about putting robots / noindex in this file, but really didn’t want to confuse Googlebot so just left it blank since the 410 would hit first.

<html lang="en">
<head>
  <meta charset="utf-8">
</head>
<body>
</body>
</html>

The sitemap.xml file

Again, not totally necessary. Some SEO’s recommend rebuilding the old sitemap.xml file from the original site and submitting to Google Search Console. I chose not to, and had great results.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>https://example.com/</loc>
<lastmod>2019-12-20T10:46:35+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>1.00</priority>
</url>
</urlset>

An extra step:ย allow me to put on my tinfoil hat for a second. If you really want to send Google a signal that your site has changed, point your domain to a new host (and new IP) then stick your robots, sitemap, .htaccess, and index.html on that server. This is just a theory that it may help Googlebot see a drastic change in your site’s infrastructure, it can’t hurt you.

Remember, the above steps are to remove your website from Google Search Results and does not apply if you are moving your website. If you are moving your website(domain) there are a completely different set of steps to take.

Also note that if you take the above steps verbatim, you will null and void any kind of 301 redirects you might have in your .htaccess file.

Any questions, please feel free to hit me up on Twitter or if you need help with your website you can always contact us.

Thank you!

Patrick Coombe
Hello I'm Patrick Coombe and I'm the CEO and Founder of Elite Strategies, an agency I started in 2009. One of the main reasons I love blogging about SEO is the research it takes to come up with the posts. It allows me to not only write about what I love, but to learn more about the industry in the process. I hope you enjoyed this post, if you did consider sharing it or even better linking to it!
  0 COMMENTS
Blog SEO Web Development WordPress

Leave a Reply

Your email address will not be published. Required fields are marked *

Contact Info

2240 Woolbright Road Suite 206
Boynton Beach, FL 33426

Phone: 561-526-8457
E-mail: info@elite-strategies.com
Fax: 561-526-8707