Lessons of the Trade : Simple way to deter web scrapers

If your website is a target of web scrapers (http://en.wikipedia.org/wiki/Screen_scraping), here’s a simple way to keep them on their toes..

Every few days (or weeks) make a small change to the web page format. The trick is to make the change in such a way so that your users do not notice it or are not annoyed by the change. Most of the simple scrapers use, screen scraping and this would confuse the hell out of them 🙂 and usually deters the most abusive ones. The smart ones would have approached you in the first place, requesting permission to scrape or get authorized for an interface.