SMotW #69: incident root causes

Information Security Metric of the Week #69: proportion of information security incidents for which root causes have been diagnosed and addressed


'Learning the lessons' from information security incidents is the important final phase of the incident management lifecycle that also involves preventing, detecting, containing and resolving incidents.  Its importance is obvious when you think about it:
"Progress, far from consisting in change, depends on retentiveness. When change is absolute there remains no being to improve and no direction is set for possible improvement: and when experience is not retained, as among savages, infancy is perpetual. Those who cannot remember the past are condemned to repeat it."
George Santayana

This week's example metric picks up on three crucial aspects:
  1. Root causes must be determined.  Addressing the evident, immediate or proximal causes of incidents is generally a superficial and unsatisfactory approach since problems upstream (e.g. other threats and vulnerabilities) are likely to continue causing trouble if they remain unidentified and unresolved.
  2. Diagnosis of root causes implies sound, competent, thorough analysis in the same way that doctors diagnose illnesses.  Casual examinations are more likely to lead to misdiagnoses, increasing the probability of failing to identify the true causes and perhaps then making things even worse by treating the wrong ailments, or implementing the wrong treatments.
  3. Addressing root causes means treating them appropriately such that, ideally, they will never recur. The fixes need to be both effective and permanent.

Before you read ahead, think for a moment about what we've just said.  Given the positive nature of that analysis, you might be tempted to implement this metric immediately ... but systematically applying the PRAGMATIC criteria reveals a number of concerns:

P
R
A
G
M
A
T
I
C
Score
85
85
67
40
77
40
48
16
40
55%

Aside from the undeniable Costs of analysing in depth and fully addressing root causes, it seems there are issues with the metric's Genuinness, Accuracy and most of all its Integrity ratings.  

One of the ACME managers who scored this metric expressed concern that the people most likely to be measuring and reporting the metric (meaning ACME's information security professionals) would have a vested interest in the outcome.  While hopefully such professionals could be trusted not to play political games with the numbers, the fact remained that they are actively involved in determining, diagnosing and addressing root causes, hence there is a distinct possibility that they might be mistaken, especially given the practical difficulties in this domain.  Information security incidents often have multiple causes, including contributory factors (such as the corporate culture) that are both hard to identify and difficult to resolve.  They may well believe that they have eliminated root causes whereas in fact even deeper issues remain unaddressed.

Given the promising introduction above, the metric's disappointing 55% score led ACME management to put this one on the watch list for now, preferring to implement higher-scoring metrics in this domain first.  The CISO was asked to think of ways to address the independence and trust issues that might put this metric back on the agenda for ACME's next security metrics review meeting.