A universal KPI

For January's security awareness module on ISO27k, I'm developing a detailed checklist with which to assess, evaluate and score each of the information security controls recommended by ISO/IEC 27002 (as summarized in Annex A of ISO/IEC 27001)*.

The checklist/scoring format is one I invented years ago and have been using and refining ever since. It is a kind of maturity metric that has proven very valuable in practice, giving surprisingly consistent and useful results despite the subjective nature of the checks.

I am laying out 4 'indicators' for each control from '27002, specifying the kinds of things that would typically correspond to scores of 0% (exceptionally weak or missing controls) through 33% and 67% to 100% (exceptionally strong or cutting-edge controls). The 50% centre point on the scale divides 'inadequate' from 'adequate' controls, although that only really applies in the context of a mythical generic mid-sized organization with minimal information risks and hence security requirements. For many commercial organizations, 60% may be a more appropriate target, varying between organizations and controls - e.g. a financial services organization is likely to have more substantial information risks and hence needs stronger controls to ensure confidentiality, integrity and availability of information, than a typical manufacturing or retail business; an engineering design firm may value data integrity above all else, given the health and safety implications and liabilities if its output is inaccurate.   

Looking back over the draft checklist, I've noticed that the scores for most controls correlate with 'assurance' activities. At the top end, 100% scores often involve strong assurance measures such as thorough, independent audits by competent auditors. At the bottom end, assurance measures are conspicuously absent: if it's not painfully obvious already, even a cursory check would no doubt reveal that the controls are either completely absent or totally inadequate, but checking simply isn't performed at the 0% level - in fact, it probably doesn't even occur to those involved. 

In the middle ground, assurance activities either drive systematic improvements where necessary, or increase confidence that the controls in place are sufficient - fit for purpose, of decent quality, doing a good job.

Therefore, assurance appears to be a universal KPI, a Key Performance Indicator that would be applicable and valuable to almost any organization that seeks to measure and improve the quality and maturity of its approach to information risk and security management. 

Assurance is an over-arching control on a higher conceptual plane than most information security controls. The benefits of assurance include:
  • Checking to ensure that the right things are being done, and things are being done right;
  • Investigating and evaluating things, digging deeper than otherwise occurs and challenging the status quo;
  • Hopefully generating credible evidence to demonstrate or prove that, making it possible for the organization's management, owners and other internal or external stakeholders to increase their confidence and trust that the organization is soundly governed, managed, operated and controlled;
  • Generating insight such as improvement suggestions, as a result of the investigation, analysis and discussion arising;
  • Spreading good practices, especially if those performing assurance activities are highly experienced and competent across a broad range of industries, organizations and situations.
So, becoming good at assurance drives the organization to a better place.

Other universal KPIs might also be relevant to information risk and security, such as:
  • Oversight - the middle and junior management equivalent to assurance, watching over, guiding and monitoring activities in a more hands-on fashion;
  • Information risk management practices, especially within a systematic, structured framework such as ISO27k, SP800-53, COBIT or NIST CSF incorporating information security management, incident management, compliance management and business continuity management as well;
  • Measurement practices - the very act of focusing on stuff that is important enough to be worth measuring tends to achieve improvements, hence the importance of designing/selecting and implementing appropriate metrics (including sensible KPIs by the way!);
  • Formalization, for example policies, procedures, guidelines, awareness and training all being managed proactively as a coherent and coordinated suite of activities that business people find beneficial rather than sheer red tape;
  • Compliance - involving both reinforcement of required practices and enforcement of the rules, which implies the need for clearly defined rules and the associated checking and motivational activities (assurance again).

    * I'm aware that not all of the ISO/IEC 27002 controls may be applicable to any organization, and that other controls may be required - in fact, I'm using ISO 22301 as a guide to business continuity controls in place of '27002's pathetic section 17, and I may use CSF or other standards on cloud controls to supplement/extend '27002 section 15. The scoring checklist needs to be considered, adapted and applied sensibly according to the context ... but, trust me, it's much easier, quicker and more effective to start with this guidance than a blank sheet!