You may not realize it but in websites all over the world, backlinks are rotting faster than a honey tomato in the bottom drawer of your refrigerator.
To kick things off, let’s consider this scenario:
Your favorite mommy blogger posts an article on her blog: she writes about her favorite new diaper genie alternative, posts a video of a baby and a cat fighting over a toy. The next week, the product she linked to is removed and the YouTube video is taken offline due to a copyright violation.
With just one post, that website now has several “broken links” and has lost some SEO value. This is link rot in its purest and most simple form.
What exactly is link rot?
The vast majority of websites link out to other websites to form what we call “backlinks.” Over time, the sites that are linked to are taken off line, go down, change their name, or something else causing a 404 or what we like to call “link rot” or “backlink rot.”
A more simple definition of backlink rot: a gradual deterioration of the external and internal links that a website links to over time.
Links can rot for a variety of reasons:
- pure laziness (don’t feel like replacing bad links)
- lack of knowledge / ignorance
- lack of finances (not enough money to renew domains)
- improperly configured websites
- accidental misconfiguration
- amongst others
Whatever the case may be, links are constantly rotting all over the internet. It is up to us to try and fix this!
Why is link rot bad?
One of Google’s primary SEO ranking signals is internal and external link structure. Google’s search algorithm “scores” websites based on the pages they link to internally (such as their about us page, contact page, or blog), as well as the pages they externally link to (such as wired.com, Huffington Post, or the New York Post.)
Over time as backlink rot occurs and links start to go down, redirect or 404 Google starts to notice this and may or may not devalue the page where this is happening, maybe even your entire domain.
According to NPR over 50% of the links are dead from the Supreme Court Opinions and over 70% of links from Harvard Law Review no longer work! In another audit, half of the links were dead already in Supreme Court opinions, and for the Harvard Law Review, just over 70 percent of the links there don’t work anymore.
Some links (and websites in general) are more prone to link rot than others.
Another reason why backlink rot is bad, or sub-optimal is because it is generally bad for user experience. What happens when you visit a site, click on a link and see a 404? Most likely, you are frustrated or annoyed because you wasted your time and can’t find what you are looking for.
If you look closely at Google’s Webmaster Guidelines, many of their recommendations align closely with user experience (UX.) For instance, doorway pages, spammy text, and broken links are all “bad” according to Google but also happen to also be poor user experience examples as well.
In short, link rot is generally frowned upon because:
- it can cause confusion / annoyance
- bad for UX in general
- could lead to ranking issues
- lost customers, they might go somewhere else
Dealing with backlink rot
Unlike wood rot in a home, you don’t need to rip out your entire infrastructure in order to fix link rot. The best way to deal with backlink rot is to do a full backlink audit of your website. For this, my personal recommendation for Windows users is Xenu which is a tool I’ve been using for almost 10 years now. It is a great tool and one of the few SEO tools that I stand behind.
For WordPress users, there are a number of plugins you can install such as Broken Link Checker that will scan your site for broken links. While this link checker is great, it isn’t perfect. You really have to know how to properly configure it. For instance if you set the timeout setting too low, you’ll wind up with false positives / negatives.
WordPress has a plethora of plugins to help avoid broken links and link rot
After you’ve scraped your entire website and have a proper accounting of your backlinks, start making a list. I personally like to put everything in Excel, but to each their own. After you’ve got working list, take some time to reflect upon your list to look for any patterns:
- are there reoccurring domains that appear on multiple pages?
- are there certain file types (HTML vs PHP) that are 404ing?
- does there appear to be mainly broken redirects vs 404?
- do the majority of your broken links exist during a certain time frame?
By tracking patterns you can make the task of repairing these broken links much easier. For instance, if you have a re-occurring domain that is a broken link on your website it probably isn’t a broken page, the website is most likely broken completely.
If you find patterns in your broken links, it might be easy to fix these large chunks of links.
If you have a small site (under 1000 pages) or only have a few dozen broken links, it is definitely worth your time to do a manual spot check on each link, meaning:
- open each link in a web browser
- follow it to its destination
- check to see if it is broken, redirected or missing
- make determination based on answer
For instance, some smaller hosts might have forgotten to pay their hosting bill while you ran the check. If that is the case you might want to wait a few days until they pay and the link will be back. Or there might be a small issue preventing the site from loading.
Another example is when we did a recent backlink audit on our company website, we noticed there was a ton of “broken” links to Moz when we did a scan with our tool. Turns out they weren’t really broken, Moz made some major changes in the last few years that affected all of their backlinks:
- they migrated from seomoz.com > moz.com
- they switched from https:// to https://
Both of these changes caused issues for our external links, but were easy fixes. While most of these links were auto-redirected, it is my personal policy to make sure all links take the path of least resistance.
Treating link rot is more of an overall attitude / policy than just running a tool or audit. Yes, running a backlink audit every so often will surely fix most of your backlink rot but being proactive about it will minimize the link rot more over time.
There are also two ways of looking at backlink rot:
- fixing the broken links on your own website i.e. internal links(missing about us page)
- fixing the broken links that your website links to i.e. external links (a link pointing to an article that no longer exists)
Whatever the case may be, it will behoove you and your users to take a look at this.
Being Proactive: Avoiding link rot
Here are some tips that I’ve compiled that will help you avoid link rot over time:
Avoid linking to anything that you think might go down, or be taken down. For instance it might not be a good idea to link to a comment thread on reddit, those threads often get deleted.
Always link to the primary article source. For instance if BBC News ran a major story but an aggregator picked it up, link to the BBC news.
Always be aware of URI / permalink structure – e.g.
https://www.example.com/my-news-story.html vs https://www.example.com/my-news-story.html?=reffered_by_patrick
Avoid link shortening services, esp fly by night ones. For instance linking to “bit.ly” or “goo.gl” is much safer than linking to a link shortener such as “pat.rick” or something along those lines. Shorteners such as Bit.ly and goo.gl are supported by large conglomerates and supporters whereas private ones are more prone to technical difficulties, domain renewals or other issues.
Think twice before you choose not to renew a domain name or delete your own content. You could be affecting other websites that are linking to it. In my experience it is always best to contact a competitor before you delete content or a domain, and see if they’d be interested in purchasing it from you.
Link Rot and Link Building
Here is a way to turn something horrible into something great. Find articles that are ranking for keywords you want to rank for, run backlink checks and see if you can find any broken links in them. Better yet you can do the reverse and find competitors that are linking and look for broken links in there.
Once you have a list of broken links, you can swoop in with a better version of your own article and kindly ask them to link. This isn’t any kind of secret, and really it is a very common SEO tactic. I suppose this is just a variation of Brian Dean’s famous “Skyscraper Technique” but it has been around as long as I can remember.
Another variation and one that I really love is by Ravan Vavrilas of Cognitive SEO who writes about using the growing problem of Wikipedia link rot and leveraging it for links and traffic. Only thing I’d like to add to this method is that if you are going to “fix” broken Wikipedia links, make sure you replace them with quality links, you don’t want to get your account audited. You also don’t want to put all your eggs in one basket, or all your links in the same Wikipedia account.
The future
If you or your agency is looking for a great digital cause that you can donate to, please consider donating to archive.org. Archive.org does its best to archive the entire web, as often as possible. So for all of those websites and links that go down, archive.org tries to take a digital snapshot of that website as it existed. They are a non-profit that relies on donations, so every little bit counts. In 50 years from now when you want to look back at “old times” this is the company that is going to make it happen.
You can browse archive.org using “The Wayback Machine” online. Here is an example of this website’s archive from March of 2011 and then another from 2013. My, how things have changed.
As we said earlier, treating backlink rot is more of an attitude than a quick fix. You must constantly be aware of the content that you produce, redirect and delete. You must also be aware and be responsible for the links that you create all over the web. One reason why blackhat SEO is so frowned upon by Google, is that it creates a huge mess of broken backlinks and backlink rot throughout the web.
As a webmaster, you are responsible for backlink rot on your website. If you are a small business owner and don’t have a webmaster, it is imperative that you partner with someone that can help you sort through this.
A few resources
The Wikipedia page on Link rot has some excellent information
A great discussion on Hacker News from late 2015
The NPR story referenced already in this post
Get Social!