Earlier this month, NSA director Keith Alexander quoted a staggering statistic from antivirus firm McAfee in a speech: that the cost of cybercrime around the world amounts to $1 trillion a year. Trace the methodology behind that often-repeated number, however, and you may be left hoping that the math geniuses at the NSA are better statisticians than the sources of their factoids.
In a report earlier this week, non-profit media outlet Pro Publica published an investigation into that $1 trillion figure as well as another from Symantec pegging the cost of cybercrime at $250 billion a year. While the Symantec stat led to a circular wild goose chase, the McAfee number’s credibility evaporated much more quickly: When Pro Publica contacted the outside security experts who had worked on the three-and-a-half year old study (PDF here) for McAfee, they disclaimed any responsibility for the trillion-dollar figure and said it was added to the press release about the study without their knowledge.
Now McAfee has contacted me to explain: No, the statistic was not simply made up. Yes, it’s just a “ballpark figure” and an “unscientific” one, the company admits. But despite Pro Publica’s criticisms and its own rather fuzzy math, the company stands by its trillion-dollar conclusion as a (very) rough estimate.
“This is a non-scientific number that’s been taken out of context and misused,” says McAfee spokesperson Ian Bain, who also points out that the figure refers to total data loss, including accidental breaches, not just cybercrime. “We worked closely with the researchers and sponsored the report. But we needed an additional piece of analysis, a total figure. So we talked about the best way to extrapolate that. That’s where we got $1 trillion.”
Here’s how McAfee came to its 12-zeroes conclusion: It hired the consultancy Vanson Bourne to perform a survey of about 1,000 senior IT executives, asking them about their data losses and the value of the lost data. That led to average reports by country of revenue losses from breaches, which averaged out to $4.6 million per company, varying from $375,000 per firm in the U.K. to $7.2 million in China.
But after those numbers had been reviewed and a report written by a team of security experts and academics led by Purdue’s Center for Education And Research in Information Assurance and Security, (CERIAS) McAfee decided to extrapolate from the numbers further for its press release on the study, to try to estimate the total cost of data loss for the world. Its researchers found statistics from chambers of commerce and censuses that counted the total number of businesses in six of the 12 countries in the study, Bain says, and multiplied the averages Vanson Bourne had found by those numbers.
For the other six countries where it didn’t have business data, it eyeballed their population size and GDP, and roughly doubled the total, Bain told me. The result was close to $1 trillion.
In other words, McAfee used a sample size of around 1,000 executives to extrapolate to many millions of businesses–based on the U.S. census it counted close to 30 million businesses in America alone–and then extrapolated again to the millions of global businesses it couldn’t count.
“We took the estimates and built out from them,” says Bain. “I think what we did was very reasonable.”
It’s not much of a surprise that Eugene Spafford, a Purdue professor who worked on the study, doesn’t agree. “We were able to make statements in the report based on the data, but in terms of having enough confidence to do reasonable extrapolations, it just wasn’t there,” Spafford says. “They had told us early on they wanted to get overall magnitude. We pushed back, because the measurement of these losses is very difficult and not necessarily reliable.”
Spafford points to biases in the data like the fact that recipients of the survey who didn’t respond weren’t counted, and may have been less likely to have experienced a costly data breach. “People are notoriously bad about detecting and valuing these losses,” says Spafford. “They overestimate how much they lose, or the total lifetime loss of the information, and that makes the totals look far larger.”
I pointed to a case McAfee highlighted, the theft of $300 billion military plans for an F-35 jet. But Spafford argues it’s difficult to attribute all of that loss to any single year. “You have to ask, did that require ten years of research? And how much would it really cost to recreate? Those are the kinds of questions that would need to be evaluated in coming up with a total,” he says.
- Page 1 / 2