PRAGMATIC business cases

Build better business cases with PRAGMATIC metrics

Imagine, purely for the sake of argument you understand, that your management is distinctly ambivalent towards, say, security awareness*.  A few limited security awareness activities are tolerated, typically for compliance reasons and generally 'under sufferance' rather than because anyone actually believes they are necessary, oh no.  Funding is all bar non-existent: there is no 'security awareness budget' as such, although activities such as security training are funded from general training budgets belonging to HR, departmental slush-funds, or (as a desperate last resort) self-funded by employees.  Most 'awareness activities' are unplanned and unintended consequences of information security incidents.

OK, you get the picture.  

Now imagine you are an information security pro who appreciates the potential for security awareness.  As far as you personally are concerned, awareness is more than merely worthwhile: it's a crucial part of the organization's security framework.  Without awareness, you might as well all give up and go home.

So, rather than admit defeat, you resolve to Do Something Positive About It.  Your challenge, then, is to persuade management to take a fresh look at security awareness, to appreciate what it can do for the organization, and in due course to put their money where their mouth is.

Question is, how?

The near-universal approach in situations of this nature is to develop some sort of proposal for a security awareness program, perhaps a budget request or a business case.  The idea is to build and document a persuasive argument, sufficiently positive and credible (relative to other proposals on the table) that management allocates the requisite resources and gives it the nod.  In our experience in many different organizations, this is generally the way that sizable (meaning potentially costly) projects and activities are initiated*. 

Again, how are you going to construct your case?

The basics are straightforward: if you are not the CSO or CISO, start by running your idea past them, partly to gauge their interest and partly to make sure that nothing you do will conflict with whatever cunning plans they are quietly cooking up.  Be gentle with them - inept managers sometimes view bright subordinates as more of a challenge than an opportunity.  Hand them a rough outline proposal if it helps, and give them some time and space to latch on to (or 'steal') the idea.  Trust us, their overt support - or lack of it - will become crucial later.

Go to Finance for a template/skeleton budget request or business case, preferably also a worked example or a real one that was considered and approved.  That will give you cues such as the overall structure and format, length, style, level of detail, types of financial analyses and so forth - all important stuff that you can easily pick up yourself, perhaps with further guidance or assistance from those nice people in Finance, since a standardized approach makes things easier for them and for management.

Moving ahead, develop your unique content, putting meat on the bones.  This is where you explain your ideas for security awareness, specifically, suggesting how the awareness program should be structured, managed and funded.  

At some point, you will project the costs and benefits of the program, implying that you first need to understand what they are.  This is where the PRAGMATIC approach comes into its own.  There are many different ways to assess costs and benefits, hence many possible financial metrics.  Which one is the best?  In the book's example metrics chapter, we PRAGMATIC-scored and compared popular metrics such as ROI/ROSI, NPV and IRR in the information security context.  They each have their pros and cons, but the PRAGMATIC scores (for ACME Enterprises Inc) turned out to be very similar, which is handy really as your choice may be determined by corporate convention or accounting policy.  

Aside from the financial criteria, you will no doubt wish to explore various other aspects of the proposed awareness program, including parameters that are not expressed (directly at least) in dollars.  If you find yourself short of ideas, brainstorming the pros and cons of different awareness approaches can be a very creative team exercise, especially if you have the luxury of a group of bright, sparky colleagues with sufficient interest in the topic to get actively involved.  Using collaborative working techniques such as the Security Metametrics Forum, you can also exploit the wider social network of professional colleagues and peers, including other business people with an interest in information security.  The choice of metrics is an excellent excuse to probe your colleagues from Risk Management, HR, Legal/Compliance, Finance, Operations, IT and other parts of the corporation for their take on security awareness.  What is important to them?  What aspects interest or concern them the most?  What would they just love to know, if only it were possible?

Pretty soon, you'll amass a jumble of ideas and suggestions for your putative awareness program, some of which will equate directly to obvious metrics while others might take a bit more thought.  The key point is to think deeply about how you might measure various parameters of the program.  Coming up with specific metrics is ideal, but the thinking process behind it is arguably even more valuable.  It forces you to be reasonably precise about what you are trying to measure (the scope of the metric), why it needs to be measured (the purpose and rationale) and, in time, how you are going to measure it (the design).  

[The question of how to measure stuff is almost incidental, by the way.  If you are new to security metrics, you probably have a narrow view of what's even possible, and may feel that some of the most interesting facets of the awareness program are literally unmeasurable.  If you have done a degree in math/statistics, or indeed any scientific subject, you will hopefully recall the wonderful variety of measurement techniques used in research.  Read any metrics book, and our previous blog items,  for lots more prompts.  In almost all cases, there are several different ways of measuring things, hence your pile of ideas grows ever taller.]

Now it's time to make sense of the jumble before the pile topples over.  Use the nine PRAGMATIC criteria to score your metrics.  If any metrics are too ill-defined and vague to assess and score, you must either firm-up your understanding or set them aside, on the assumption that they probably won't be worth measuring anyway.  If you already have a good bunch of potential metrics, this eminently pragmatic approach is perfectly valid.

Using a spreadsheet makes it easy to sort and hence rank your metrics in descending order of overall PRAGMATIC score, or by any of the individual criteria.  Metrics that score highly overall are strong candidates to include in your business case.  Lower-ranking metrics that score relatively well on both Predictiveness and Relevance to information security may also be worth considering, although you may need to address whatever factors depressed their scores, for example by changing the nature or design of the metrics.

Finally, you should have enough material to make a convincing argument, writing a well-rounded business case or proposal that not only makes bold claims about what it hopes to achieve, but suggests specific metrics by which its success or failure will be judged.  All that introspection, brainstorming, peer-consultation and analysis has a subtle pay-off at this stage, since you will inevitably have thought through the underlying issues in some depth.  It's a bit like the concept of test-driven development: if you know precisely how an IT system will be tested and accepted into production, designing and building the system to pass those tests is more straightforward than if the tests are only to be figured out later on.  In other words,

The map is more useful if you
know where you're headed 

Please let us know if you use this approach.  We'd love to know how it turns out for you, especially if you find any wrinkles or have any improvement suggestions for others who might be willing to give it a go.

* Although we've picked on security awareness for this piece of bloggery, it could equally have involved a proposal to invest in an Intrusion Prevention System, a Public Key Infrastructure, or a replacement for the antiquated and hopelessly inadequate spreadsheet-based system currently used to mismanage system access rights (!).  In fact, it needn't even be information security related at all: the basic principles and issues are much the same for any kind of corporate investment that is not classed as a must-have - and to be frank, even those investments that are considered no-brainers would benefit from better metrics.  There are generally different ways of Doing What Has To Be Done, some being more cost-effective than others.  If you understand the success criteria well enough to propose explicit metrics, chances are you're on the right track.  We'll continue this train of thought later.