Management awareness paper on authentication metrics

User identification and authentication (I&A) is a key information security control for all systems, even those that allow public access (unless the general public are supposed to be able to reconfigure the system at will!). As such, it is important to be sure that I&A is working properly, especially on business- or safety-critical systems, which in turn implies a whole bunch of things. I&A must be:
  • Properly specified;
  • Professionally designed;
  • Thoroughly tested and proven;
  • Correctly implemented and configured;
  • Used!;
  • Professionally managed and maintained;
  • Routinely monitored.
Strangely, monitoring is often neglected for key controls. You'd think it was obvious that someone appropriate needs to keep a very close eye on the organization's key information security controls, since (by definition) the risk of key control failure is significant ... but no, many such controls are simply implemented and left to their own devices. Personally, I believe this is a serious blind spot in our profession.  

If unmonitored key controls fail, serious incidents occur unexpectedly. In contrast, management has the opportunity (hopefully!) to spot and respond to the warning signs for key controls that are being routinely monitored and reported on using suitable metrics.  Security metrics, then, are themselves key controls.

The management-level awareness briefing paper briefly sets the scene by outlining common requirements for I&A. It then briefly describes four types of metric that might be used to monitor, measure and generally keep an eye on various aspects of the control. Perhaps the most interesting is the authentication failure rate ... but to be honest my thinking on metrics has progressed in the 7+ years since this paper was written. The metrics in the paper look naive and basic to me now. Since I'm updating the authentication awareness module this month, I'll be thinking up better I&A metrics when I rewrite the paper for customers ... perhaps even scoring them using the PRAGMATIC method ... oh and revising those awful graphics!