The one point graph

Given my interest in metrics, I'm always on the lookout for statistics relevant to the monthly topics to incorporate into and illustrate our awareness materials. It's hard, though, to find credible figures that we are prepared to pass along to our customers. There are plenty of numbers tossed around but few of them have any substance - at least not enough to satisfy my admittedly rather cynical inquiry.

Take this paragraph, for instance, by Bill Taylor-Mountford, lifted from one of the many marketing blogs promoting companies sponsoring the RSA conference: 

"When ransomware took centerstage a few years ago, we failed to anticipate its magnitude and severity. 2016 was the year when ransomware dominated headlines as it exploded to become one of the biggest security wakeup calls for CXOs. That year the FBI estimated that ransomeware could be a $1 billion source of illicit income for cyber criminals, and, a survey by Osterman Research showed that 39 percent of organizations in some of the world’s largest superpower countries were hit by a ransomware attack. Other reports show that Asia Pacific suffered more than 10 million ransomware attacks in the first half of 2016 alone."

So according to Bill, ransomware "exploded" in 2016, as indicated by:

• The FBI estimating that ransomware "could be a $1 billion source of illicit income for cyber criminals". In the course of preparing this month's awareness module on ransomware, I hunted for the source of that $1bn estimate. All I have been able to find is a throwaway comment ascribed to an unnamed FBI source by a journalist in the cited CNN news piece under a dramatic headline "Cyber-extortion losses skyrocket, says FBI" which, in turn, cited an earlier CNN piece "'Ransomware' crime wave growing". Tucked away in there we find this snippet: "The FBI says it received 2,453 complaints about ransomware hold-ups last year, costing the victims more than $24 million dollars." Presumably someone has extrapolated from $24m in 2015 to $1bn in 2016. Wow! The $1bn figure has been widely repeated ever since as if it was a hard, proven fact. It is easy to think up a random number while, thanks to the nice round figure "a billion bucks" having a certain cachet, it has spawned an Internet meme, fueled by no end of individuals, journalists, marketers and organizations repeating it ad nauseam;
• The cited Osterman Research survey was conducted on behalf of an antivirus company (the same company, I notice, that features prominently in those CNN pieces ...) and used by them for marketing purposes, hence I would be surprised totally amazed flabbergasted gobsmacked if the survey was not designed to demonstrate how valuable their antivirus software is, for example grossly inflating the size of the problems they claim to solve. Furthermore, they surveyed about 500 IT managers in American, Canadian, British and German companies - none it seems in the Asia Pacific region that Bill was specifically blogging about. There are other issues with the survey, typical of its kind. We aren't told how the sample population was selected for the survey but there's a fair bet it consisted of current or prospective customers of the antivirus company - most likely a self-selected sample of people willing to click a few buttons on a web survey, not a random sample - which thereby invalidates many statistics and calls the whole thing further into question;
• Bill ended the paragraph by referring tantalisingly to "other reports", making no attempt whatsoever to identify them. We cannot assess the claimed "more than 10 million ransomware attacks [in Asia Pacific] in the first half of 2016". For all we know, that's another figure plucked out of thin air. Personally, I try hard to discount such unsubtantiated claims out of hand, although something once read cannot be un-read so it is another little piece of tripe festering at the back of my brain - and yours too, I'm afraid, since even if you didn't catch the original blog, I've repeated it here. Sorry, I'm part of the problem! Hopefully that mental image of festering trip is powerful enough to trump the statement.

The emotive term "exploded" clearly implies that the ransomware problem was much less before 2016, and yet Bill fails to substantiate that. This smells a lot like one of those one-data-point graphs: place a dot somewhere on the graph, connect it to the origin at bottom-left by a line, and project it forward as far as you like. In reality, we can only guess where the Y-axis crosses the X-axis, and as for the nature of the relationship and hence the linearity of the line, well with only the one limited data point to go on, good luck defining anything since an infinite variety of possible lines pass through that single point. Worse still, as I've been saying, the one data point itself has a distinctly dubious pedigree and doesn't qualify as "data" (not even under Doug Hubbard's deliberately liberal interpretation). Hence whether the problem "i" in 2016 in any meaningul, literal sense looks like mere speculation to me, hyperbole designed to fill column-inches. 

"Oh but it was informed speculation" I hear you thinking out loud. "It came from the FBI!".  As we Kiwis say, "Yeh, nah."

I am prepared to accept that ransomware is a current issue, fair enough. It has certainly received quite a lot of media coverage in the past year or so, if not actually 'dominated the headlines'. There are credible but largely apocryphal stories about organizations [mostly fairly small public bodies - hmmm, strange that] being coerced into paying a few thousand dollars each to decrypt their data. But as to whether ransomware "exploded" to a '$1bn problem', no we won't go that far, not yet anyway: let's wait for at least one more data point before graphing the trend, eh? Let's see how it turns out.