As anyone who operates or frequents web-based bulletin boards and blogs knows, spam messages have spilled out from the realm of e-mail and are invading message boards like giant locusts from a Bert I. Gordon movie. The reason these (often automated) spammers are out searching for boards is very simple: Google and other search engines rank pages by the number of sites that link to them, and if enough fake posts are made pointing to a spam site, the page may appear in the top page of results for legitimate searches. The other problem is spam blogs, where spammers create fake blogs with information stolen from legitimate sites, in order to boost their search ranking.
Microsoft Research has been working on the problem and has figured out a way to use the spammers' methods against them. Their tool, entitled Strider Search Defender, automates the search for spam messages by taking a context-based approach that looks for URL redirection, a tactic often used by the spam sites and fake blogs.
The site has a neat diagram of how the spamming process works. Both legitimate message boards with spammed posts and the fake spam blogs blast enough links to the spammer's site to get high rankings on Google and other search engines. Then, when the user clicks on the link, they are directed to the site, which is then often redirected to whatever the spammer really wants to sell you.
Spammers use these redirection techniques, combined with cloaking (a method of presenting different content to the search engine crawler than is generated when a real user visits the page) to get high rankings for legitimate searches while presenting the user with ads for typical spammy things like fake Viagra. The Microsoft Strider Search Defender checks for both of these techniques using heuristic analysis. To get past cloaking, the program launches a copy of a web browser and loads the questionable URL, then tracks what the browser renders. To reduce false positives, the program uses a whitelist of known legitimate advertising syndicators.
Yi-Min Wang, the researcher heading up the project, gave an example of its use. He gathered more than 17,000 URLs on BlogSpot and fed them into the URL Tracer. The group was able to identify the top 25 target-page domains that were behind the Google-hosted spam messages, even when they were hidden behind a series of redirects. Wang identified the top six as s-e-arch.com, speedsearcher.net, abcsearcher.com, eash.info, paysefeed.net and veryfastsearch.com, which together were responsible for 45 percent of the BlogSpot URLs.
The tool has already helped remove junk results from MSN Search. Hopefully Google will jump on this bandwagon as well, cutting the spammers off before anyone can see them. Of course, they will probably just find another, sneakier way to make our lives full of mystery meat.