It's hard to underestimate the power of modern data-mining techniques, which can sift through massive databases and identify unexpected correlations, providing new insights and the raw material for further research. For example, the availability of extensive data from hospitals enabled an analysis that suggests that local privacy laws may hinder the adoption of electronic medical records. But these studies rely on the existence of the raw data in electronic form; even when data's available, it may be spread across multiple sources and locked in a variety of incompatible formats. All of that explains the significance of an announcement by OpenSecrets.org, which is making a large collection of data related to political activities publicly accessible.
OpenSecrets.org is run by the Center for Responsive Politics, which appears to be largely funded by charitable foundations. The Center has been in operation for over 25 years, and it claims to provide an independent, nonpartisan view into the influence of money on the US ppolitical system. It runs OpenSecrets.org to provide a clearing house of data on political contributions and lobbying efforts, with the hope that bringing some transparency to the inner workings of the political system will lead to a more informed citizenry.
For the most part, it does this by acting as a data aggregator. In recent years, governmental requirements have meant that members of the different branches of government have had to provide details on their finances, both personal and for their campaigns. The same goes for groups, such as those that perform lobbying or have focused on issue advocacy (often termed 527 organizations). All of those filings, however, have had distinct formats, and were published by different branches of the government, making it difficult to come to terms with it all. OpenSecrets.org drew conclusions about trends and specific behaviors in part by pulling together data from these disparate sources and analyzing it as a coherent whole.
Apparently, behind the scenes, the group has developed a process for importing the data from these disparate sources and placing it in a structured database. The new announcement suggests that the group now considers this process production-ready. Anyone who agrees to its terms of service and sets up an account can use the data under the Creative Commons Attribution Non-Commercial Share Alike license.
The data can be accessed either in bulk (downloaded in CSV format) or through a series of Web services with a documented API. OpenSecrets.org has used the latter to produce a series of embeddable Web widgets that allow people to track donations, their sources, and the money flowing into specific races. In addition to data on campaign finance and lobbying efforts, the APIs provide access to the personal financial disclosures of elected and administration officials; a resource table ties it all together by defining the terms used throughout the data.
"Putting our data into more hands will put more eyes on Washington and, we hope, engage more Americans in their government," said Sheila Krumholz, the Executive Director of the Center for Responsive Politics, in announcing the release.
The fact that all of this data was available somewhere means that a sufficiently motivated journalist or political operative probably could have come up with the information were it useful for their needs. So, the new resource may make it easier for someone to find embarrassing information on a politician, but it's not unlikely to unleash a new wave of revelations about the messy side of political financing.
What the data may provide that's new, however, is the ability to identify broad patterns in the data that aren't apparent by looking at the information that can be obtained from any one source, or by focusing on a single campaign. Although the name of OpenSecrets.org suggests a focus on individual disclosures, it's possible that, by opening its data to the public, the group will enable researchers to develop a clearer sense of the big picture of money and politics.