IT security spend as a % of IT budget

According to an article in The Register, Gartner has pointed out that 'proportion of IT budget spent on IT security' is not a good metric.

One can determine any metric's strengths and weaknesses systematically and objectively using the PRAGMATIC method, so here goes:

• Predictiveness: at a superficial level of analysis, the budget obviously affects the amount that can be spent on, or invested in, anything, hence there is bound to be some relationship between the money spent and the amount achieved ... but that is not a direct, linear relationship (in practice a somewhat vague correlation I suspect). Organizations with tight budget constraints have to spend more carefully, and naturally focus their efforts on optimizing the value they obtain. Furthermore, many would acknowledge the preponderance of snake oil salesmen in the IT security field, hence spending more might even, in some cases, be counterproductive. Score: 50%
  
  
• Relevance: the metric may be relevant to IT security, and to financial management for the organization, but is that enough to score well on this criterion? What of its relevance to information risk and information security, compliance, governance and business continuity? Score: 75%
  
  
• Actionability: at first glance, increasing or decreasing the proportion of the IT budget allocated to IT security would be the obvious response to low or high values of this metric. However, that's not how budgets are normally determined. Conceivably the metric might be one of the factors taken into account in the budget proposal. More likely, management would expect to see a reasoned, rational business case to spend money, not something as crude as a proportion of spend (even if it was presented as a benchmarking comparison relative to other similar organizations - assuming that could be done). Score: 55%
  
  
• Genuineness: does the numeric value of this metric genuinely and straightforwardly reflect the object of measurement? Could it be manipulated, perhaps by someone with a hidden agenda? The metric is generated very simply by dividing one number by another, so there's not much leeway for manipulation ... but can those two numbers - the base data - be trusted? I'm not so sure (more notes below). Furthermore, issues with the true meaning of the metric (the next PRAGMATIC criterion) may be explained away by creative interpretation when presenting the metric, especially if the audience is unfamiliar with broader, more mature concepts such as information risk. Score: 65%
  
  
• Meaning: the meaning of any metric depends on the intended audience. It's a matter of perspective. So who is or are the prime audiences for this metric? For financially- and/or IT-aware managers, the metric seems self-evident. To other general managers, it may also appear meaningful, at face value, but to anyone who digs a little deeper, and most likely to the CISO, ISM or other experts in risk, security, governance, compliance etc. (including, reportedly, the analysts responsible for Gartner's report), the metric is distinctly misleading. There is more to managing information risk than IT security, and anyway the amount of investment in IT security is not necessarily reflected in the results. It is depressingly easy to come up with examples of IT security investments that have not paid off, including some that have failed spectacularly: consider Target, Sony and others for instance. Score: 30%
  
  
• Accuracy: Gartner acknowledged that the metric varies widely between organizations, in the range 1-13%. Does IT security status even vary by such an amount between organizations, let alone vary in accordance with the value of this metric? Possibly, yes, but personally I doubt it. There are other significant concerns over the accuracy (see below). Score: 10%
  
  
• Timeliness: it takes hardly a moment to calculate this metric provided the base data are available - simply divide the two figures. Obtaining the base figures is straightforward too, assuming the organization captures or reports them as part of the budgetary/financial management. Score: 95%
  
  
• Independence: could someone (such as an auditor or manager) validate the metric? Yes, checking the calculation is trivial but there is some question about the base data, and how they are determined (see below). Score: 65%
  
  
• Cost-effectiveness: although the cost of generating this metric is negligible, its value is not strong. Consequently, and especially if compared to many other similar metrics, this one does not generate much net value for the business. Score: 30%
  
  
• Overall PRAGMATIC score = 53% (a simple arithmetic mean of the individual scores)

By all means challenge the thinking, adjust the individual scores and even weight the individual criteria if you feel so inclined. According to my PRAGMATIC analysis, inaccuracy and low net value are the most significant issues with this metric, along with its potential to mislead naive recipients. There are questions about the base values from which the metric is calculated, and about the relationship between the metric and the organization's IT security status, plus still bigger questions about its relevance to information risk and to the organization's business objectives. The overall score of 53% is near the bottom of the 'acceptable' scoring range (just above the 50% cutoff point), making this metric barely acceptable - hardly worth considering unless there is no better metric (which I am sure there is - 'IT security maturity' is one such shining example). 

That said, the metric's PRAGMATIC score could be improved if those and other issues were addressed by refining it ... which I leave as an exercise for the keen reader.

Gartner clients can obtain the report (cited as "Identifying the Real Information Security Budget" in The Register but identified as "How to Manage and Defend Your Security Budget" on the Gartner page to which they linked) for free. Others must part with $195 for the pleasure of reading the 10-page report if this blog piece and a moment's quiet reflection was insufficient. As to whether the $195 represents good value for money, and whether it legitimately qualifies as 'IT security expenditure', I leave to your discretion. I'll simply point out that it prompted the journalist to comment on, and then me to scratch beneath the surface of, what turns out to be a commonplace but lame metric.