THE winner takes all, it is widely supposed in computing circles. Indeed, geeks have coined a word, “Googlearchy”, for the way in which search engines encourage web traffic towards the most popular sites. The belief that search engines make popular websites ever more popular, at the expense of other pages, is now being challenged by research.
The apparently magical ability of search engines such as Google to return relevant websites even when given the sketchiest of clues by the person entering a query relies on the use of mathematical recipes or algorithms. Google works by analysing the structure of the web itself. Each of its billions of pages can link to other pages and can also, in turn, be linked to by others. If a page is linked to many other pages, it is flagged up as being important. Furthermore, if the pages that link to this page are also important, then that page is even more likely to be important. The algorithm has been made increasingly complex over the years, to deter those who would manipulate their pages to appear higher in the rankings, but it remains at the heart of Google's success.
Google is not alone in this. Many search engines take account of the number of links to a website when they return the results of a search. Because of this, there is a widespread belief among computer, social and political scientists that search engines create a vicious circle that amplifies the dominance of established and already popular websites. Pages returned by search engines are more likely to be discovered and consequently linked to by others.
Not so, according to a controversial new paper that has recently appeared on arXiv, an online collection of physics and related papers. In it, Santo Fortunato and his colleagues at Indiana University in America and Bielefeld University in Germany claim that search engines actually have an egalitarian effect that increases traffic to less popular sites.
The researchers developed a model that described two extreme cases. In the first, people browsed the web only by surfing random links. In the second, people only visited pages that were returned by search engines. The researchers then turned to the real world. They plotted the traffic to a website—measured as the fraction of all page views made in a three-month period—against the number of incoming links made to that website. To their surprise, they found that the relationship between the two did not lie between the extremes suggested by their model but somewhere completely different. It appears to show that the supposed bias in favour of popular pages is actually mitigated by the combination of search engines and people following random links.
The paper, which was posted on arXiv for comment, has now come under attack. Matthew Hindman, a political scientist at Arizona State University, says that the data used in the research are pretty shoddy. Moreover, he says, the discrepancy between the model and the real world does not necessarily come from the role of the search engine.
Whether Dr Fortunato's thesis stands the test of time remains to be seen. That it is tested must be a good thing.