Management awareness paper on insider threat metrics

How do you measure 'insider threats' in your organization?  If your answer is "We don't!", then I have to wonder how you are managing insider threats. 

Without suitable metrics, how do you figure out how much of a problem you might have from employees, contractors, consultants, temps and interns?  How do you determine where best to spend your security budget? How do you persuade management to loosen the purse strings sufficiently to address the risks?  I guess you guess!

The discussion paper breaks down 'insider threat' into chunks that can be measured sensibly.  The main divide falls between deliberate attacks (such as frauds by insiders) and accidents (such as mistakenly overwriting the entire production database - don't laugh, it happened to me 25 years ago and the nightmare still haunts me today!). 

The paper picks up on one of the most productive sources of information security metrics: the IT Help/Service Desk's problem and incident management process.  Most mid-to-large organizations these days route employee queries and problem reports through a centralized helpdesk function that acts as a clearing house, offering first-line support and routing unresolved calls to the appropriate resolving agencies for specialist help.  A valuable core part of their role is to ask questions and record information about calls in the ticketing system, tracking the calls through to completion.  The system, then, is a goldmine of information about queries, problems, events, incidents and other issues, including "near misses" (assuming the organization is sufficiently clued-up to encourage workers to report close-shaves as well as actual incidents).

If this is all Greek to you, find a spare hour or two to speak to a helpdesk supervisor about the call-ticketing system, about how calls are categorized and recorded, and how to squeeze suitable reports out of the system ... but, before you get completely carried away on a wave of enthusiasm, planning how to use all that sexy statistical and textual information, think very carefully about what you are doing.  The availability of information is not, in fact, the main concern when designing security metrics.  A far more important issue is to figure out what you want to know, what questions need to be answered.  Conceiving questions that fit the available answers is putting the cart before the horse.

Management awareness paper on network security metrics

Measuring network security involves, first and foremost, determining what 'network security' encompasses, and how it relates to the business.

Writing way back in 2007, we said that network security "comprises a range of technical and procedural controls designed to prevent, detect and/or recover from security incidents affecting the corporate data networks – incidents such as unauthorized access (hacking), worms and other malware infections, and unplanned network downtime". The context for the paper was a security awareness module exploring security arrangements protecting data networks against both deliberate and accidental threats.

The paper described ways to measure network security incidents, controls, risks, compliance and governance.  It ended with an upbeat conclusion and call-to-action: "Do not neglect the value of having the experts present and discuss reports with management.  The dialogue that ensues adds value to the written reports.  Why not present and discuss these ideas with your management and seek their opinions, bringing to the table some prototype reports in one or more formats to stimulate discussion and clarify their objectives?"

The PRAGMATIC approach is a structured, systematic way to consider the pros and cons of various metrics, leading ultimately to a decision on which ones (if any!) to adopt.  Just as important as the final destination, the method leads managers and infosec pro's together on a journey of discovery.

My guess is that the network security incident and compliance metrics would probably score above the others proposed in the paper but I can't tell for sure because I don't know a thing about your situation or measurement needs: your evaluation may well come to a markedly different conclusion.  Furthermore, in discussing the metrics paper, you might come up with 'variations on a theme', meaning variants of the metrics proposed that would score more highly, and perhaps something completely different, especially if it turns out that none of the metrics in the paper are suitable.  

That spark of creativity is the real power of PRAGMATIC.  Scientifically analyzing the factors determining the strength and suitability of the metrics under discussion leads to a better understanding of the metrics and of the measurement needs.  Contrast that with the blank-sheet approach.  Faced with the question "What network security metrics shall we adopt?", most business managers would be at a loss to know how to proceed.  If the security, network or IT people suggest one or more technical security metrics to fill the void (perhaps plucked from the air or picked out of some banale list with a pin), the managers don't have the basis for evaluating them, meaning that they are quite likely to accept whatever is offered, perhaps later coming to realize that they aren't exactly ideal.  If the metrics truly don't work, it's back to the drawing board to pick some more ... assuming someone has the good sense to call a halt to the senseless waste of time and effort (that's not a facile comment: many organizations drift aimlessly along for years with inappropriate, unsuitable or low-value metrics because everyone assumes someone else must be using them!*).

Although it is possible for the organization to develop a decent set of metrics by sheer trial-and-error, there is a distinct chance that good metrics will be discarded or discounted for arbitrary reasons, and that opportunities to 'tweak' metrics the better to fit the business needs will be missed.  Using the PRAGMATIC method as a systematic process to select, develop and maintain suitable metrics has to be a better way, don't you think?

* That reminds me: have you audited your organization's security metrics? It may be a tedious assignment but a competent and diligent auditor should be able to follow the paper trail from gathering the source data through analysis and reporting to the decisions arising - if any.  If the trail goes cold, that's a strong indication that the metrics are simply not working, implying not just a waste of effort and money but an information void and lost opportunity.  Low quality metrics are a costly distraction, literally worse than useless!

PCI embraces security awareness

The PCI Security Standards Council's Security Awareness Program Special Interest Group has released an 'information supplement' to PCI-DSS, suggesting an awareness approach that is remarkably similar to ours.

Best Practices for Implementing a Security Awareness Program is a well-written guide elaborating on four key ideas:

1) Security awareness is a vital tool supporting the business. "It is therefore vital that organizations have a security awareness program in place to ensure employees are aware of the importance of protecting sensitive information, what they should do to handle information securely, and the risks of mishandling information." [We go further in emphasizing the business value of information security, for example giving management confidence that information assets will be sufficiently well protected when exploring new business opportunities.]

2) Security awareness is best delivered on a continual basis, all-year-round. "Security awareness should be conducted as an on-going program to ensure that training and knowledge is not just delivered as an annual activity, rather it is used to maintain a high level of security awareness on a daily basis." [Absolutely! Our monthly deliveries are explicitly intended to support the continuous, rolling approach to awareness.]

3) Security awareness should address three audience groups, namely all personnel (with the aim of making security a habit), plus managers and personnel in 'specialized roles' meaning those who work directly with card holder data (both of whom need additional information/guidance). [Our three streams address personnel in general, managers and specialists with a professional interest in information security.]

4) Security awareness content needs to be delivered through a variety of suitable mechanisms throughout the organization, implying that over-reliance on a single channel (such as email, posters or the intranet) is less than ideal. [Our monthly modules deliver 20-30 different formats of material, all consistent, covering the same subject area, at the same high level of quality, and fully customizable. Our customers have enormous flexibility in their approach to awareness, while we support awareness programs at every stage of maturity from the basics to the most advanced.].

The remainder of the guide is not quite as impressive. It goes on to suggest the awareness topics, diving into the specific information requirements relating to PCI (alone). "It is recommended that general security training for all personnel include defining what constitutes cardholder data (CHD) and sensitive authentication data (SAD) and the organization’s responsibility to safeguard both. A high level overview of the importance of the PCI DSS may also be included; to ensure personnel fully understands the purpose behind an organizational policy to safeguard cardholder data. To ensure all personnel are engaged stakeholders in the security awareness program, the roles and responsibilities of all staff to protect CHD and SAD should be outlined during all security awareness training, in accordance with organizational policy." The guide mentions need to protect all forms of information (not just computer data, media and systems), to address social engineering, and to make personnel aware of policies and procedures. [We cover more than just PCI. We take a far broader perspective on information security because many information assets (not just credit card details!) are highly valuable to the organization, and all of them deserve or require adequate protection. Furthermore, the organization's information security-related compliance obligations go well beyond PCI-DSS. As far as we and our customers are concerned, PCI is just one of many obligations - merely the tip of the iceberg.]  

The guide also suggests some metrics, divided into two groups. The "Operational metrics" groups appears to be a random and incomplete assortment of security control objectives (labelled "Metrics") and measures (labelled "Training effectiveness indicators"): neither are much good, in my opinion, since they don't directly and obviously relate to business objectives. For example, "Reduced system downtime and network or application outages" is clearly a technical/IT objective, whereas the corresponding business objective (presumably something like "Highly available IT systems and networks suporting critical business activities") remains unstated, while the suggested measure "Consistent, approved change-management processes; fewer malware outbreaks; better controls" is merely another incomplete set of technical objectives. The other group, "Training program metrics" barely even relate to the objectives of the awareness program stated earlier in the guide. All in all, I highly recommend skipping the metrics section completely. [Read my book PRAGMATIC Security Metrics instead!] 

There are a few references in the guide for further information. Despite us having been doing this stuff consistently since 2003, I can understand them not even mentioning us but it is very disappointing that Rebecca Herold's bible on security awareness was not cited ... which seriously leads me to wonder about the Special Interest Group which produced the guide. Surely someone among the august group of 100 or so organizations has read Rebecca's book and seen the light? Or are they all still floundering along in the dark?  :-)

Management awareness paper on malware metrics

Malware - malicious software - encompasses a variety of computer viruses, Trojans, network worms, bots and other nasties.  

Malware has been the scourge of IT users ever since the Morris worm infected the early Internet way back in 1988.  Despite the enormous global investment over the intervening years in information security controls against malware (including security awareness!), it remains a significant security concern today. 

Although antivirus software companies sometimes admit that they are fighting a losing battle, malware is generating so much income both for the VXers (malware authors) and their criminal masterminds, plus the antivirus software companies, that the arms race looks set to continue for the forseeable future.  Both sides are constantly investing in new tricks and techniques, fuelling a thriving black market in zero-day exploits and novel malware.

Meanwhile, the rest of us are lumbered with paying for it in one way or another. Addressing the malware risk is more involved than simply throwing money at antivirus software: malware metrics are the key to understanding the balance of risk against control, investing wisely, and doing our level best to keep up with, if not stay one step ahead of the game in this dynamically evolving situation.

Download our third security metrics awareness paper here.