Stopping the Splogs

Spam-blogs (or Splogs) are becoming as annoying as spam email.

Sploggers set up sites with zero content, and just a bunch of ads. Then they spam the comments threads on popular web sites, and link back to their splog. This tricks Google into thinking that the splog is site with relevant information. This boosts the Google page rank of the splog, which means more ad revenue for sploggers.

Well... nuts to that!

People have come up with lots of techniques to block sploggers. Requiring registration, making people type in the letters they see on the screen, or typing in answers to simple math problems. All good.

This weekend, I came across a very interesting technique for stopping splogs. It goes beyond the basic tricks and eliminates the value of splog comments entirely. If everybody followed this technique, splogs would never work in the first place.

For those not familiar with how the Google page rank works, the best way to boost your rank is to get bigger web sites to link to yours. That's it. Google rank is rarely a technical problem: its an information architecture and public relations problem. Its a people problem.

Sploggers manipulate the open comment policy of large sites to make more links to their splogs, bypassing any authorization. Heck, some of the first search engine optimization firms just hired teams of cheap engineers to flood sites with bogus comments.

The fix is simple: never allow links to external sites! Instead, make all external links point to a redirect page on your site. For example, if your site was mysite.com, and some splogger made a link to othersite.com, you should munge the URL to be somthing like this:

http://mysite.com/redirect?url=othersite.com

The link above is technically still in mysite.com, even though it redirects out to othersite.com. By redirecting, all links on your site merely point back to your site. Nobody else's page rank is affected. Its a nifty little trick to cut off access to sploggers.

Its slightly more complex than this, however. You need to set the correct 'no-follow' headers, or some future search engine may try to follow JavaScript redirects. Also, you will need to encode the URL for better security, otherwise phishers may use your site to make an evil URL look legitimate.

Naturally, the side effect is that relevant links in comments are ignored along with splog links. If a commenter sincerely knows of a relevant site, Google will not mark it as such. However, this isn't much of a problem. The author of the original story can update the original story and add any relevant links to the end.

The original article was from 2003, and I'm surprised that more sites don't use this trick. The only other one I saw using it was amazon.com, in their new Wiki feature. Perhaps the negative effects outweigh the positives for large blog sites... or perhaps these sites use anti-spam filters on the comments.