Micro vs. macro metrics

Whereas "micro metrics" focus-in on detailed parts, components or elements of something, "macro metrics" pan out to give a broad perspective on the entirety. 

Both types of metric have their uses.

Micro metrics support low-level operational management decisions. Time-sheets, for example, are micro metrics recording the time spent on various activities, generating reports that break down the hours or days spent on different tasks during the period. This information can be used to account for or reallocate resources within a team or department or identify. Normally, though, its true purpose is to remind employees that they are being paid for the hours they work, or as a basis on which to charge clients. 

Macro metrics, in contrast, support strategic big-picture management decisions. They enable management to "see how things are going", make course-corrections and change speed where appropriate. The metric "security maturity", for example, has implications for senior managers that are lost on lower levels of the organization. I have a soft spot for maturity metrics: they score strongly on the PRAGMATIC criteria, enabling us to measure complex, subjective issues in a reasonably objective and straightforward fashion.

The sausage-machine metrics churned out automatically by firewalls, enterprise antivirus systems, vulnerability scanners and so forth are almost entirely micro metrics, intensely focused on very specific and usually technical details. There are vast oceans of security-related data. Lack of data is not a problem with micro metrics - quite the opposite.

Some security professionals are 'boiling the ocean' using big data analytics tools in an attempt to glean useful information from micro metrics but a key problem remains. When they poke around in the condensate, they don't really know what they're looking for. The tendency is to get completely lost in the sea of data, constantly distracted by shiny things and obsessing about the data or the analysis ... rather than the information, knowledge, insight and wisdom that they probably should have gone looking for in the first place.

It's like someone stumbling around aimlessly in the dark, hoping to bump into a torch!

Just as bad, when a respected/trusted metrics "expert" discovers a nugget and announces to the world "Hey look, something shiny!", many onlookers trust the finder and assume therefore that the metric must be Good, without necessarily considering whether it even makes sense to their organization, its business situation, its state of maturity, its risks and challenges and so forth ... hence they are distracted once more. As if that's not enough, when others chime in with "Hey look, I've polished it! It's even shinier!", the distractions multiply. 

The bottom-up approach is predicated on and perpetuates the myth of Universal Security Metrics - a set of metrics that are somehow inherently good, generally applicable and would be considered good practice. "So, what should we be measuring in security?" is a very common naive question. Occasionally we see various well-meaning people (yes, including me) extolling the virtues of specific metrics, our pet metrics (maturity metrics in my case). We wax lyrical about the beauty of our pet metrics, holding them up to the light to point out how much thy gleam and glint. 

What we almost never do is explain, in any real detail, how our pet metrics help organizations achieve their objectives. We may describe how the metrics are useful for security management, or how they address risk or compliance or whatever, but we almost invariably run out of steam well before discussing how they drive the organization towards achieving its business objectives, except for a bit of vague hand-waving, cloud-like. 

By their very nature, it is even harder to see how micro metrics relate to the organization's business objectives. They are deep down in the weeds. Macro metrics may be up at the forest canopy level but even they are generally concerned with a specific area of concern - information security in my case - rather than with the business.

I guess that's why I like the Goal-Question-Metric approach so much. Being explicit about the organization's goals, its business and other high-level objectives (e.g. ethical or social responsibility and environmental protection), leads naturally into designing macro metrics with a clear business focus or purpose. 

Popular posts from this blog

Pragmatic ISMS implementation guide (FREE!)

Image
Early this morning ( very  early!) I remotely attended an ISO/IEC JTC 1/SC 27/WG 1 editing meeting in London discussing the planned revision of ISO/IEC 27003:2017 . Overall, the meeting was very productive in that we got through a  long  list of expert comments on the preliminary draft standard, debated the objectives of the project and the standard and reached consensus on most points. In summary: 27003 is to be revised to align with the current 2022 releases of ISO/IEC 27001 , 27002 and 27005 : These changes are  mostly  minor aside from the new section 6.3 on ISMS changes.
Read more

Two dozen information risks that ISO forgot

Image
Selecting the wrong controls - controls that are inappropriate, ineffective, too costly, impracticable, fragile, unnecessary, counterproductive or whatever, often as a result of blind faith in fads and fashions of the day and FOMO e.g. MFA, AI, cyber Failing to select the right controls - controls that are ideal for the particular situation, both now and in perpetuity, for whatever reason - mostly ignorance and prejudice Selecting and implementing controls at the wrong time or in the wrong way (where 'wrong' includes ineffective, inappropriate, sub-optimal e.g. bolting on controls rather than designing and building them in) Inept and inaccurate identification, analysis and quantification of risk, including reliance on p oor quality (incomplete, inaccurate, out of date, misleading, unreliable ...) information about actual risks, particularly subtle and emerging risks plus those involving deliberate concealment and misdirection e.g. fraud, misinformation, disinformation, propagan
Read more

Philosophical phriday - compliance risk

Image
According to a vendor's promotional video interview I saw recently, the 'cybersecurity compliance burden' has allegedly become so significant that [customer] organisations are eagerly buying [their] software tools and services to help them manage and fulfil their obligations. The vendor's argument goes that, instead of accumulating a ragtag bunch of policies and other controls relating to user Identification and Authentication (I&A), for instance, it makes sense to:  Identify all the cybersecurity-related laws, regulations and standards that apply to the organisation; Examine them for any security control requirements relating to, say, I&A; Rationalise the I&A controls down to the smallest set that satisfies all the requirements - the lowest common denominator; Design, implement, use, manage and maintain those I&A controls; Have the I&A controls checked or audited to gain assurance that the compliance requirements are met.  OK so far? Sounds reasonab
Read more

ISMS internal audit priorities

Image
A thread on the ISO27k Forum sparked my imagination over coffee this morning. Hope had previously asked for assistance with an ISO/IEC 27001:2022 audit plan.  Bhushan offered a lengthy and generally sound response explaining how to use a spreadsheet with tabs to plan and record the audit work performed on 100% of the main body clauses and 50% of the 93 Annex A controls, day-by-day. That's OK ... except it wasn't entirely clear that he was interpreting and elaborating on the standard's actual requirements. ISO/IEC 27001 does not explicitly require, for example, that (as Bhushan stated) "ALL the management system clauses from 4 to 10 AND their sub-clauses need to be listed and audited" in an ISMS internal audit, although evidently he interprets it in that way. In clause 9.2.1, the standard states a requirement for internal audits to provide information on whether the ISMS conforms to the organization’s own requirements for the ISMS plus the requirements of the stan
Read more

Reading between the lines of ISO27001 [L O N G]

Image
ISO/IEC 27001 is a succinct, formally-worded standard for two key reasons: It is deliberately generic, being applicable to all manner of organisations regardless of difference in location/s, size, industry, maturity, structure, information risk and security status ... and so on. In effect, it specifies the lowest common denominator - the things that ALL organisations should be doing to manage their information security controls, as a minimum. The hurdle is set low enough that every organisation ought to find value in designing, implementing and operating an I nformation S ecurity M anagement S ystem as laid out in the standard. It is a certifiable standard, explicitly specifying the characteristics that every certified organisation's ISMS is expected to have. Again, it is a minimal specification with no concept of typical, average or maximum security: that is entirely down to the organisations themselves to determine, following the information risk management processes minimally de
Read more

Passionate dispassion

Image
Someone who is actively involved in, or is managing, an activity is patently not independent of it. They may well make a conscious, rational and determined effort to be objective, dispassionately reviewing evidence etc ., but their subconscious/emotional biases/prejudices and beliefs/value-systems will inevitably influence what they do. With the best will in the world, they will struggle to challenge and assess their past decisions and activities, especially if they were "certain" or "determined" or genuinely believed they were "doing the right thing". Furthermore, it is very hard for anyone to review the things they did not do, decisions they did not make or options they did not even consider. Mostly, they remain out of sight or out of the question.
Read more

45 ISO Management Systems Standards

Image
The ISO website  currently lists 45 published M anagement S ystems S tandards: 1. ISO 7101:2023 Healthcare organization management — Management systems for quality in healthcare organizations — Requirements 2. ISO 9001:2015 Quality management systems — Requirements 3. ISO 10012:2003 Measurement management systems — Requirements for measurement processes and measuring equipment 4. ISO 13485:2016 Medical devices — Quality management systems — Requirements for regulatory purposes 5. ISO 14001:2015 Environmental management systems — Requirements with guidance for use 6. ISO 14298:2021 Graphic technology — Management of security printing processes
Read more

Philosophical phriday - a noncompliance ramble

Image
In a previous philosophical phriday post , I moaned about vendors of security compliance support/management tools and services over-promising and under-delivering - an admittedly biased, even cynical opinion piece about the compliance imperative . A recent article in Corporate Compliance Insights notes that "CISOs are not just defenders against cyber threats but also champions of compliance and operational resilience". Hmmm, are CISOs 'compliance champs', really? Today, I'm discussing alternatives to being compliance-driven. How else can organisations drive their information risk, security and related concerns in a positive direction?
Read more

Adaptive SME security Crowdstrike special

Image
As if on cue, along comes a golden opportunity to consider what the Adaptive SME security  approach has to say regarding the Crowdstrike incident: That's not 20/20 hindsight but foresight: I've picked out the most relevant rows from the security controls table published in the guide 24 hours before the incident.  Although Crowdstrike primarily supplies much larger enterprises than SMEs, the incident could equally have afflicted other security software, or indeed operating systems such as Windows and assorted cloud apps commonly used by SMEs. Regardless of the details, it is a wake-up call, an opportunity to consider and respond to the information risks ... and to adapt , accordingly.
Read more