We’ve ragged on the MPAA and RIAA for bad studies before, but it never occurred to us that credit card companies would use crappy survey methods to determine cybercrime. After all, legally speaking, whenever you get defrauded, they’re the ones on the hook: federal law dictates you’re only liable for up to $50 in fraudulent charges. You’d think they’d want to be accurate about this.
But, apparently, every cybercrime estimate you’ve heard is completely full of crap, at least according to Microsoft’s research arm.
Honestly, what they uncovered is a bit flabbergasting. Instead of actually totaling up losses to cybercrime, which you think would be the logical way to do it, instead, many figures were arrived at by…telephone surveys.
“First, losses are extremely concentrated,” they wrote in the report, “so that representative sampling of the population does not give representative sampling of the losses. Second, losses are based on unverified self-reported numbers. Not only is it possible for a single outlier to distort the result, we find evidence that most surveys are dominated by a minority of responses in the upper tail.”
Translated from the Statistician, that means one sucker who really thought a Nigerian prince needed his help knocks the whole survey out of whack.
Worse, many of the surveys didn’t bother to verify the data, meaning that it might not even be a sucker — just some guy who wanted to prank the survey taker. The whole paper is a fascinating read about just what goes into your Internet scare sausage, and a valuable reminder that some people just really suck at their jobs.
(Image courtesy bfishadow on Flickr)