SMotW #29: controls coverage

Security Metric of the Week #29: security controls coverage



Baseline Defenses Coverage (Antivirus, Antispyware, Firewall, and so on)
This is a measurement of how well you are protecting your enterprise against the most basic information security threats. Your coverage of devices by these security tools should be in the range of 94 percent to 98 percent. Less than 90 percent coverage may be cause for concern. 
Whereas Jaquith's metric involves simply determining the proportion of IT systems that are running security software, we had in mind a more sophisticated metric that takes into account a wider range of security controls - perhaps a comprehensive review or audit of information security controls in use across the enterprise against a standard such as COBIT, ISO/IEC 27002 or the Information Security Forum's Standard of Good Practice.

Although we feel it would be quite Predictive and Relevant to information security, the overall PRAGMATIC score for our version of the metric is mediocre, let down by the ratings for Accuracy, Timeliness, Independence and Cost:

P
R
A
G
M
A
T
I
C
Score
87
89
65
40
74
35
46
40
30
56%




A common issue with crude 'coverage' metrics is that they generally gloss-over important details and hence do not necessarily reflect the actual information security risks in different situations.  For example, a storeroom full of new PCs and servers waiting to be configured and installed would presumably depress the metric if they were not running firewalls, antivirus etc., yet the risk to the organization is negligible.  On the other hand, a single critical network server with something like an out-of-date antivirus package or a misconfigured firewall might legitimately be assessed as having full coverage, whereas in fact it represents a substantial risk.  

This issue (a metric risk) can be addressed if the people doing the measurement take such factors into account, but their interpretation increases the subjectivity of the process.  This in turn affects the Accuracy, Timeliness and Independence scores, while the Costs increase as a result of needing skilled people to assess coverage in more depth.