How many metrics?

While perusing yet another promotional, commercially-sponsored survey today, something caught my beady eye. According to the report, "On average, organizations track four to five metrics".  

Four to five [cybersecurity] metrics?!!  Really?  

Oh boy.

Given the importance, complexities and breadth of cybersecurity, how on Earth can anyone sensibly manage it with just four to five metrics? It beggars belief, particularly as the report indicates that three quarters of the 1,200 surveyed companies had at least a $billion in revenue, and more than half of them have at least 10,000 employees. With a total cybersecurity expenditure of $125billion (around 80% of the total global estimate), these were large corporations, not tiddlers.

The report indicates the corresponding survey question was "Q30. Which of the following cybersecurity metrics does your organization track, and which metrics are the most important?". Well OK, that's two questions in one, and 'the following cybersecurity metrics' are not stated.

Having been quietly contemplating that one remarkable, counter-intuitive finding for about an hour, I've thought up a bunch of potential explanations so far:

1. The four to five cybersecurity metrics are just those considered 'key' by the CISOs and other senior people surveyed.
2. The four to five are just the respondents' choices from the 16 metrics presumably offered in the question (we aren't told what metrics were offered in the question, but there are 16 listed in the report).
3. Cybersecurity is not being managed sensibly.
4. Cybersecurity is not being managed.
5. Cybersecurity is not what I think it is - a neologism for IT security or more specifically Internet security protecting against deliberate, malicious attacks by third parties.
   
6. CISOs and the like haven't got a clue what they are doing.
7. Most CISOs and the like chose not to answer the question (of the 1,200 companies surveyed, we aren't told how many respondents answered this or indeed any other question: perhaps they were getting bored by question 30 of an unknown total).
8. CISOs and the like simply lied, for some reason, or their responses were inaccurately/ineptly recorded.
9. The word 'track' in the question strongly implies that the four to five metrics are measured and reported regularly, showing trends over time. Other metrics that are not 'tracked' in this way were not noted.
   
10. The survey was ineptly designed, conducted, analysed and/or reported.
11. The survey was non-scientific, biased towards the interests of the commercial sponsors (who, presumably, offer 'solutions' measured by the chosen metrics ...).
12. The survey company is blatantly circulating misinformation, designed to mislead.
13. I am misinterpreting the phrase. Perhaps 'On average' or 'metrics' mean something other than what I understand. 
14. Perhaps 'four to five' is a transcription error: maybe the count was forty-five.
15. I'm totally mistaken: it is possible to manage cybersecurity by tracking just four to five metrics. The finding is valid. I need to readjust my head.
    
16. I'm seriously over-thinking this, putting far too much emphasis on those eight words taken out of context.

Of that list, while I'm happy to discount the patently ridiculous possibilities, I find it hard to choose between the remainder. I'm drawn inexorably back to something I have complained about previously here on the blog: I suspect that the report is merely another marketing exercise, not a properly designed and conducted scientific study. I find it lacks credibility and integrity, is untrustworthy, and hence is not worth any more of my time, or indeed yours - so I refuse to provide a link to the source.

 

4-0-4  Move along, nothing to see here.