Management awareness paper on network security metrics

Measuring network security involves, first and foremost, determining what 'network security' encompasses, and how it relates to the business.

Writing way back in 2007, we said that network security "comprises a range of technical and procedural controls designed to prevent, detect and/or recover from security incidents affecting the corporate data networks – incidents such as unauthorized access (hacking), worms and other malware infections, and unplanned network downtime". The context for the paper was a security awareness module exploring security arrangements protecting data networks against both deliberate and accidental threats.

The paper described ways to measure network security incidents, controls, risks, compliance and governance.  It ended with an upbeat conclusion and call-to-action: "Do not neglect the value of having the experts present and discuss reports with management.  The dialogue that ensues adds value to the written reports.  Why not present and discuss these ideas with your management and seek their opinions, bringing to the table some prototype reports in one or more formats to stimulate discussion and clarify their objectives?"

The PRAGMATIC approach is a structured, systematic way to consider the pros and cons of various metrics, leading ultimately to a decision on which ones (if any!) to adopt.  Just as important as the final destination, the method leads managers and infosec pro's together on a journey of discovery.

My guess is that the network security incident and compliance metrics would probably score above the others proposed in the paper but I can't tell for sure because I don't know a thing about your situation or measurement needs: your evaluation may well come to a markedly different conclusion.  Furthermore, in discussing the metrics paper, you might come up with 'variations on a theme', meaning variants of the metrics proposed that would score more highly, and perhaps something completely different, especially if it turns out that none of the metrics in the paper are suitable.  

That spark of creativity is the real power of PRAGMATIC.  Scientifically analyzing the factors determining the strength and suitability of the metrics under discussion leads to a better understanding of the metrics and of the measurement needs.  Contrast that with the blank-sheet approach.  Faced with the question "What network security metrics shall we adopt?", most business managers would be at a loss to know how to proceed.  If the security, network or IT people suggest one or more technical security metrics to fill the void (perhaps plucked from the air or picked out of some banale list with a pin), the managers don't have the basis for evaluating them, meaning that they are quite likely to accept whatever is offered, perhaps later coming to realize that they aren't exactly ideal.  If the metrics truly don't work, it's back to the drawing board to pick some more ... assuming someone has the good sense to call a halt to the senseless waste of time and effort (that's not a facile comment: many organizations drift aimlessly along for years with inappropriate, unsuitable or low-value metrics because everyone assumes someone else must be using them!*).

Although it is possible for the organization to develop a decent set of metrics by sheer trial-and-error, there is a distinct chance that good metrics will be discarded or discounted for arbitrary reasons, and that opportunities to 'tweak' metrics the better to fit the business needs will be missed.  Using the PRAGMATIC method as a systematic process to select, develop and maintain suitable metrics has to be a better way, don't you think?

* That reminds me: have you audited your organization's security metrics? It may be a tedious assignment but a competent and diligent auditor should be able to follow the paper trail from gathering the source data through analysis and reporting to the decisions arising - if any.  If the trail goes cold, that's a strong indication that the metrics are simply not working, implying not just a waste of effort and money but an information void and lost opportunity.  Low quality metrics are a costly distraction, literally worse than useless!