Throughout my SEO career I've encountered all sorts of incredible SEO issues - from the typical (not allowed in robots.txt) to the truly bizarre (eg, a mobile-friendly site that blinds Google and displays old stupid phone version).
A surprisingly common problem is URL redirection and vulnerabilities that are rarely actively monitored and detected.
URL redirection is usually configured when the URL is modified or content is deleted.
Redirects are in place to protect the SEO performance of old URLs, make sure old URLs in Google's index don't 404 (or reach via links below), and preserve link fairness as historical links point to old URLs.
For example, in this case a very poorly constructed URL would be redirected to a good semantic URL.
Broken redirects produce errors (like 404s) or point to the wrong place. This is a particularly insidious problem because it's so hard to detect. However, it can easily be blocked.
In the case above, if those redirects Latest Mailing Database were broken, 50 external links to /women.shoes would no longer count towards that page. Page authority will drop, and so will the ranking of that page.
Since this page has less authority to distribute via internal links, this damage may also be linked to other parts of the site.
Many websites are at risk, and in most cases, the source of the problem will go undetected. The impact on organic traffic can range from mild to disruptive.
And for me, eight years ago, it was devastating.
Our story started with a disturbing drop in organic traffic
when (2008) I was working on a website in the travel industry. I came to work one morning, had coffee, had a bagel for breakfast, and used Google Analytics.
What I saw that morning was not very good.
The organic traffic decreased by about 3-4 percentage points on a week-on-week basis. It's disturbing even though my bagel is ruined.
I hope this is just one of seasonal or random fluctuations.
But not!
The drop all day went on for about a week until we were down 20%.
At this point, I'm really sick of my stomach. Like most SEO professionals, my digestive system was no worse than a drop in organic traffic for no apparent reason.
I looked up the usual suspects. The site seems fine, no SEO elements have changed. There was no drop in Domain Authority or PageRank (we were still looking for PageRank at the time). No Google updates have been announced, and no unusual conversations on SEO boards. No new competitors are stealing traffic.
I'm stuck.
I've spent weeks trying to spot any potential changes that could lead to a downturn with no luck.
Every morning I have a one-on-one with my CEO, and every day without an answer, the meeting becomes more unpleasant. About two weeks later, my lead engineer came up to me and our conversation went something like this: To the
lead
engineer: "Remember we did a site migration last year?
Me: "Yes, of course! It went well! "
Lead Engineer: "Remember the redirect we set via URL change? "
Me: "Yes, of course. Principal Engineer: "
I took them down a few weeks ago because I don't think we need them anymore. "
Me: "No"
Lead Engineer: "Yes"
Me: "No"
Lead Engineer: "Yes"
Me: "NOOOO"
Lead Engineer: "Yes". That's it for the
conversation. I'll give you the details.
I was pissed that my engineers didn't consult me first.
Really, I was pissed at myself. It's on me! I'm in charge of SEO and securing bugs.
I feel guilty. depressed. How can I let me down?
I swear I'll never be surprised again by an unplanned change.
Every legitimate link is precious
We all know how precious links are. Getting links is harder than ever. Keeping in touch should be a top priority.
How did this happen? Why are we so vulnerable?
Some history:
About the year it happened, when we moved from .Net to Rails, there was a huge company migration. I did what every competent SEOer would do: I redirected the 301 from the old URL to the new URL. This will ensure that we don't lose traffic right away, and most importantly, it will transfer authority from the old URL to the new one.
Our website migration went smoothly. In fact, traffic has increased. Everything is golden. It wasn't until a year later that those redirects stopped working.
These missing links represent almost all of the deep links on the site, roughly 30% of which point to our domain. As Google crawls and links across the internet start 404s, links in Google's index are gradually removed, our weight drops, and our traffic starts to drop.
Broken redirects are especially dangerous
. There is no good way to fix this.
l Link all work on site.
l A typical audit tool can only find live URLs and will not try to crawl old URLs.
l So far, none of the old URLs are in Google's index, so there are no dead links in search results.
Bottom line: if you're not actively testing redirects, you won't know if they're still in place.
Don't let it happen to you!
Here are some things you can do to protect your site from the potentially damaging effects of broken redirects:
l Avoid changing the URL structure as much as possible. The easiest way to prevent broken redirects is to avoid creating redirects in the first place.
l Create a good URL from the start! That way, discussions and decisions about optimizing URLs don't happen. Of course, this is not always possible, and SEO experts often inherit a sub-optimal URL structure.
l If you don't need it, don't change your URL. Of course, don't modify the URL structure to make small adjustments.
Sometimes, without options, redirects are inevitable. I can think of four clear examples:
1. The old URL structure was not well thought out and confused users.
2. Technological change requires a new structure.
3. The website is being transferred to a new domain, subdomain or folder.
4. Existing content is being moved or merged.
Ensuring Redirects Continue
If you fall into one of these categories, it is your responsibility as an SEOer to ensure that redirects are done correctly, which is an ongoing job.