Comparative security metrics

In situations where it is infeasible or impracticable to quantify something in the form of a discrete count or absolute value in specific units, comparative or relative measures are a useful alternative. They are better than not measuring at all, and in some cases easier to comprehend and more useful in a practical sense. In this respect, we disagree with those in the field who fervently insist that all metrics must be expressed as numbers of units (e.g. "20 centimetres"). It seems to us "A bit longer than a pencil", while obviously imprecise, might be a perfectly legitimate and helpful measure of something (regardless of what that thing might be - a cut on your arm for instance).

Cardinal numbers and units of measure have their place, of course, but so do ordinals, comparatives and even highly subjective measures - all the way down to sheer guesswork (and, yes, 'down to' itself implies a comparative value). Douglas Hubbard's "How To Measure Anything" is an excellent, throught-provoking treatise on this very subject.

In information security, comparisons or relations can provide answers to entirely valid and worthwhile questions such as:
  • Are we more or less secure than our peers?
  • Are we getting more or less secure over time?
  • If we both sustain our present rate of change, how long will it be before we'll surpass our competitors' level of information security?
  • Are our information risks increasing or decreasing?
  • Which are our strongest and weakest areas or aspects of security?
  • Of all the myriad changes currently occuring in information security, what are the most worrying trends?
  • Does information risk X fall within or exceed our risk appetite or tolerance?
  • Which business unit, function, department or site is the most/least vulnerable?
  • Are we spending too little, about the right amount, or too much on information security
As part of an information security awareness case study on 'the Sony hack', a management discussion paper describes three types of comparative security metrics with several examples of each.