SEO tip: prevent multiple slashes in website URLs on IIS

IIS is a highly robust and flexible web server. It allows you to use almost any URL that is technically possible. But this means your website may allow URLs which you don't want.

For example, IIS will handle URLs with extra slashes. You might find this useful, for example to honour links which have been written incorrectly. But this can have a negative impact on search engine optimisation. In this article, I'll show you a simple way to address this behaviour if it causes you a problem.

I frequently see questions posted online by people who have found that URLs with extra slashes have been indexed by search engines, typically Bing or Google, and want to correct the problem. Where possible the root cause should be addressed by eliminating links which use the wrong URL.

Example

Apple's website currently allows multiple slashes, resulting in innumerable permutations of incorrect URLs which still work, like these two examples:

http://www.apple.com//iphone/

http://www.apple.com/iphone//features/

(Apple's website runs on Apache, not IIS, but that's another story...)

Notice that these two URLs have extra slashes. As you will see if you try these URLs today, and similar examples on other websites, many websites do handle URLs with extra slashes. Unfortunately, this can lead to search engines indexing incorrect URLs and duplicate content. Multiple pages with the same content is bad for SEO. Duplicated pages detract from the original page. You can end up with the incorrect URL being indexed and the proper URL being removed from search results.

How to fix it!

There are numerous posts online by people asking how to solve the problem of double or multiple extra slashes in their URLs, and I have never seen anybody find a solution in these threads. And yet, it's easy to fix. This is why I decided to write the article!

One quick, easy solution, which I would personally reccomend, is to set-up a simple rewrite rule. ISAPI is the ideal place to intercept and fix this particular problem, because IIS passes a corrected URL to the web application, so you can't write application code to fix the problem in your web application, whether you're using ASP.NET, ASP.NET MVC, classic ASP, PHP, or any other web application framework.

Just copy and paste the following three lines into the .htaccess file or whetever you use to configure your rewrite engine. (Ironically, Helicon's own website suffers the same problem -- it's very common, but let's always strive to do better than everybody else!) This rewrite rule is tested and works perfectly with Helicon ISAPI Rewrite 3:

# Prevent double slashes in URLs, e.g. //Blog and /Home//About
RewriteCond %{REQUEST_URI} ^(.*)//(.*)$
RewriteRule . %1/%2 [R=301,L]

You can adapt a rule like this to solve a wide range of problems, and improve SEO in various ways. In fact, once you've started using an ISAPI rewrite, you'll think of plenty more useful things you can do with it.

The performance overhead of ISAPI filters is negligible. Rewrite rules are executed extremely fast and efficiently, at a low-level.

Enjoy!

24 June 2010

Share the love:

Comments: 11

Add Comment

Tim Acheson (24 Jun 10, 16:39)

As a footnote, I must say I think it's great that IIS is extensible like this. One claim of open-source proponents is that it makes it easy to add new stuff. In practice, it's rarely that easy to update or fix an open-source application. Microsoft has done an excellent job, with many products (Visual Studio is another excellent example), of showing how well-designed closed-source software can be highly extendable through APIs and add-ons. Third-party plug-ins and add-ins are an excellent way of extending software -- sometimes beyond what the original authors have time or foresight to do. ;)


Good article, 3 things:

1) Double slashes only hamper SEO if there are links connecting to that content. If there are no links then its a non issue.

2) A global rule may redirect to a 404 when it should have been a 404 in the first place! Not ideal - but probably acceptable as a global rule is safe catch all

3) Apple is big enough and has enough traffic that even if there where links with extra slashes its impact will be negligible.

Mind you your rewrite rule would work fine on Apache (the worlds most popular webserver ;) ) - See this 2008 article saying much that same as your article but with less Apple baiting (wheres the SEO juice in that??!)

http://www.mydigitallife.info/2008/07/21/redirect-or-rewrite-to-remove-double-or-multiple-slashes-in-url/

Andy Buckingham (24 Jun 10, 19:46)

Of course this ISAPI rewrite module is excellent because it's an implementation of the supplied mod_rewrite in Apache.

Tim Acheson (24 Jun 10, 20:37)

Hi Ross! Yep, I don't think Apple has too much to worry about in terms of SEO, at present. ;-) It's an age-old topic, which seems to keep surfacing. I've presented one simplified example here, but of course there's much more to it and there are various related issues.

Many websites could benefit from taking similar action, and many would find that they do have badly formed URLs indexed in Bing/Google. I implemented some basic rewrite rules to enforce correct URLs on this blog of mine long ago, but there's still a "/%5Cblog" URL in Google, probably from one dodgy link on somebody else's site.

Tim Acheson (25 Jun 10, 11:07)

Hi Andy! Yes, and to Helicon Tech's credit, their ISAPI_Rewrite behaves almost exactly like Apache mod rewrite. This makes rules configured in .htaccess beautifully interoperable between Apache and IIS. Indeed, you can upgrade seamlessly from Apache to IIS without having to change anything. ;) Of course, Helicon Tech is just one example. There's a broad variety of brilliant ISAPI products available for IIS -- that's the beauty and flexibility of IIS's API.

Tim Acheson (25 Jun 10, 11:26)

We've had some interesting topics raised here, in the comments so far! I can't resist adding my two-cents to the mix. :)

The last time I checked, back in 2007, Apache was only "14.5% ahead of IIS" and IIS was continuing to make "important gains against Apache" in market-share. Bearing in mind that Apache is free for any script-kiddie to download and use, while people still have to pay for IIS, it says a lot about the quality and success of the product that IIS has no trouble keeping-up with the competition. There's no magic at work here -- IIS is keeping pace with the competition, because it's a good product, and smart people are happy to pay a reasonable price to invest in top quality.

[I just checked the latest Netcraft Web Server Survey, which shows that IIS now has a bigger share than it did in 2000, while Apache's share is now lower than it was in 2000.]

Competition between web server platforms is an epic long-distance race of endurance, and MS keeps managing new spurts of acceleration. That's why we've seen these reports over the past few years of IIS making a "large increase" in market-penetration, as in that last link from 2002. In the end, I confidently predict MS will win the marathon. But IIS doesn't even need to win, because for the foreseeable future it'll always be in the race.

Imagine how MS's market-share might sky-rocket if they rolled-out a free version of IIS, like the "Windows Server Express" product which I have proposed. I've been begging and pleading with MS to do so, but of course there are perfectly valid counter-arguments. The MS web platform folks rightly want to preserve their revenue stream in order to invest in the fantastic improvements they keep making. And they've gone a long way toward addressing the reasons why people might wish for a free-lunch web server, with their benevolent and generous Website Spark initiative and other cunning ideas. Don't get me started on these wonderful topics...

Paul Yeager (20 Jul 10, 15:13)

Yes, nice of someone to develop a plugin so that IIS can do what Apache does natively!

Tim Acheson (21 Jul 10, 07:28)

Paul, ISAPI is obviously more flexible than a unit of baked-in functionality. This is just one example, which happens to be a URL-rewriting engine, which in turn happens to be based on Apache. (A key usecase stated by Helicon for their product is "making it possible to move configurations from Apache to IIS".) Personally, I like it, although there are alternatives which have a nice GUI; that's the beauty of ISAPI. Of course, there's a lot more you can do with ISAPI, far beyond any functionality in either Apache or IIS.


Its usefull , but only for .htaccess file(Appache), If i want to remove // from url used in asp.net application... In this case i cant able use .htaccess file because web server is different...any suggest?

Tim Acheson (12 Jul 13, 20:29)

@Arun which web server are you using? My example above is for IIS. Different servers have different ways of doing things, but whatever you do in Apache you can do on IIS.


Farmville farms even include free gift that is cedkkdfekadafadg

Tags:


  • Twitter
  • LinkedIn
  • Facebook
  • Windows Live / Messenger
  • Xbox Live
  • RSS
  • Email