Every aspect of modern science revolves around publishing important and timely articles in journals where they are more likely to be noticed--in some cases, exposure can be more important than relevance. Publishing in top tier, high Impact Factor journals maximizes the chances of having others cite your work, regardless of the publication's quality. Now, researchers have compared the citation network to a publication's page rank, and come to an interesting conclusion: the two are related, but some of the outliers show a tendency to have won their authors a Nobel Prize.
Nobel Intent's own Chris Lee recently described some of the issues caused by the modern scientific publishing process and its emphasis on the top tier journals. These journals tend to attract citations of their contents, and current measures of scientific importance rely solely or heavily on these citation numbers. The net result is that papers published in journals like Science lead to better metrics for the authors, which influence hiring, raises, tenure, and other career-altering decisions, creating a snowball effect that pushes authors toward fame and fortune.
Citation numbers (total citations, citations per publication, H-factor, etc.) while common, are generally considered to be a poor measure of scientific contribution. For example, one could generate many citations by publishing articles that turn out to be wrong and having your irate peers correct you in print. Researchers can also pad their citation numbers by selectively citing their own work or parceling out data into smaller communications.
But there are also researcher-independent issues. Citations are difficult to compare across the various scientific fields; a paper cited 20 times in some disciplines would be a failure, while similar numbers would be a resounding success in others. Citation numbers bias towards recent articles because scientific communities were smaller in the past and many archives do not include data prior to a certain date. In fact, the common joke in our lab is that nothing important happened before 1980 because our favorite database defaults to searching only post-1980 literature.
Researchers have turned to network theory to better model and understand scientific importance. A number of interesting node-based models describe papers as an interconnected web of citations, including some that rank the relative impact of journals. To a simple approximation, most of these models combine the number of links to a particular paper (the number of citations), the number of links to the citing papers, and the interconnectedness of that network with other networks of papers. It occurred to researchers at Brookhaven National Lab (BNL) and Boston University (BU) that an additional and extremely effective network analysis tool already exists: Google's PageRank.
Google began its march toward global domination (and I, for one, welcome our new Google overlords) with a simple search algorithm based on network analysis. The PageRank algorithm places a hypothetical Web surfer at a Web site (or node in a network) and assumes that the surfer will move on to a site linked at the original with a probability of 1-d. Alternatively, the surfer will not find what they want on a site and start the search over with a probability of d. When a surfer starts their search over, they are randomly placed at another node in the network. PageRank iterates through this process until there is a stable number of surfers at each site. This number is then used to rank the results for a given search term.
The researchers at BNL and BU checked the PageRank of publications of the American Physical Society and found that, generally, it scaled with the citation numbers for papers with more than 50 citations. Interestingly, they found a set of outliers where the PageRank number was substantially higher than would be expected based on citation numbers alone. You may recognize some of the authors: Bardeen, Cooper, and Schrieffer; Weigner and Seitz; Onsager; Kohn and Sham; Feynman and Gell-Mann. In fact, nine of the ten outliers were Nobel Prize-winning papers and the tenth, Cabbibo's "Unitary Symmetry and Leptonic Decays," is arguably the largest omission in Nobel history.
The researchers modified the PageRank algorithm to better reflect the search activity of scientific researchers rather than Web surfers. Generally, researchers start with a recent article and then slowly work back to older articles. To reflect this activity, the algorithm biased the starting point of searchers toward more recent articles. Also, researchers tend to search in less depth than web searchers, so the probability, d, was reduced. The resulting algorithm is called CiteRank, and can be found here.
Clearly, something must be done to address measuring scientific importance, and models based on network analysis seem to encompass far more of the reality of scientific literature than simple citation-based numbers. By using a well-established algorithm and modifying it slightly for the realities of scientific research, CiteRank appears to be a major step in the right direction. Clearly, CiteRank's ability to pick out unusually important articles is striking, but its pitfalls may only be apparent after wider adoption. For a wider perspective on measuring scientific importance and the problems with Impact Factor, check out Jonathan Gitlin's latest dispatch from ScienceOnline'09.
Journal of Neuroscience DOI: 10.1523/JNEUROSCI.0002-08.2008