Battling Referrer Spam with Wordpress

For some reason, my weblog became the target of hundreds of referrer spam hits from pornographic websites over the last week or so. I keep an eye on my referrer logs (a record of URLs that generated traffic to my site), and lately a bunch of URLs showed up which had no business being there. Some URLs are obviously pornographic, but there were one or two that looked innocent enough that when I clicked through to see who had linked to me, I got an eyeful. I really, really, don’t need that.

So, I did some research. I didn’t want to get into a trap of having to hand-modify my .htaccess file or a whitelist or a blacklist file for obvious reasons: the universe of porn and poker sites is potentially infinite. I waste enough time on this blog anyhow!

Angsuman’s Referrer Bouncer looked good, but it doesn’t play well with wp-cache. Other well-documented tricks involved endlessly modifying my .htaccess file. Bad Behavior looked good, too, but I’ve already used that plugin, and disabled it because I saw occasions where it needlessly blocked legitimate access, requiring manual intervention.

So, I settled on and installed Referrer Karma. After the painless installation (it’s not anywhere near one-click, you do have be careful and edit a file), I tested it by using one of the baddie referrers and tricking my Firefox browser to spoof the referrer, and … success! It blocked my access. Then I went to couple of my buddy-bloggers who link to me and tried to click-through and enjoyed more success. Checking the RK logfiles showed what happened: the bad referrers were added to a blacklist, and the good ones added to a whitelist.

Referrer Karma is cleverly engineered. It requires no manual intervention on my part, it does everything automatically. When a page is requested, the referring (linked) page is requested by my server, and it’s checked to see if my URL actually does appear there. Apparently, RK even requests the javascript files to be sure that my link isn’t in some javascript widget on the site, or embedded in an iframe, or anything like that. Once my URL is found embedded in the remote page, the referrer is added to the whitelist and that page need not be checked again. If my URL is not found, the referrer is blocked. Under certain conditions, the IP is blocked, too.

There is some risk that the referring IP is a webmail client or a password-protected forum. For that reason, there is an already-extensive whitelist that comes with RK, and when one of those protected sites hits a page on my blog, they just need to click on the link in the error page to pass through to my site. In one word: Nifty.

There is also some risk in slowing down my page delivery, defeating the purposes of wp-cache. I’ll have to monitor that and see if it becomes a problem. And there’s some exposure in the bandwidth department: I could be subject to a virtual denial-of-service attack just by being hit with so many new referrers that RK has to request an endless stream of pages to check. That could happen, so, I’ll have to monitor my bandwidth utilization as well.

But, all-in-all, not bad for a little research and a few minutes effort.