SMotW #59: residual risk liability

Security Metric of the Week #59: total liability value of residual/untreated information security risks

This sounds like a metric for the CFO: tot-up and report all the downside potential losses if untreated or residual information security risks were to materialize.  Easy peasy, right?

Err, not so quick, kimo sabe.

In order to report risk-related liabilities in dollar terms, we would presumably have to multiply the impacts of information security incidents with the probabilities of their occurrence.  However, both parameters can only be roughly estimated, hence the metric is subjective and error-prone which naturally cuts down on its Accuracy rating. 

The skills and effort needed to calculate the liabilities, especially with the care needed to address that subjectivity, makes this a relatively Costly security metric too, although arguably there are substantial benefits in doing the analysis, aside from the metric.  

The Actionability rating is depressed since it is unclear what management would be expected to do in response to the metric.  If the value is high, are they supposed to pump more money into information security?  And what if the value is low: is it safe to cut back on the security budget?  Either way, the metric alone does not indicate the extent or scale of the response.  There is no comparator or criterion, except perhaps for prior values, but unless you went to extraordinary lengths to control the measurement process, random variations arising from the subjectivity would generate a lot of noise masking the puny signal.

  

On a more positive note, the liabilities arising from residual risks are patently Relevant to information security, and in the form of large dollar figures, are likely to be highly Meaningful to management, given the common if crude impression of management that "In the end, it all comes down to money".  Making the effort to express information security risks in dollar terms does at least help position security as a business issue, although there are better ways.

Acme managers rated the metric's overall PRAGMATIC score a disappointing 59%, which effectively put it out of the running in its present form given that  there were several similar but higher-scoring candidate metrics on the table.  

It's not entirely obvious how the inherent weaknesses of this metric might be addressed to improve its PRAGMATIC score.  What, if anything, would you suggest?  Have you actually used a metric similar to this, and if so how did it work out?  We'd love to hear from you.