A fraudulent fraud report?



Our awareness module on digital forensics is coming along nicely. Today, in the course of researching forensics practices within organizations, I came across an interesting report from the Association of Certified Fraud Examiners. As is my wont, I started out by evaluating the validity of the survey on which it is based, and found this:
"The 2018 Report to the Nations is based on the results of the 2017 Global Fraud Survey, an online survey opened to 41,573 Certified Fraud Examiners (CFEs) from July 2017 to October 2017. As part of the survey, respondents were asked to provide a narrative description of the single largest fraud case they had investigated since January 2016. Additionally, after completing the survey the first time, respondents were provided the option to submit information about a second case that they investigated.
Respondents were then presented with 76 questions to answer regarding the particular details of the fraud case, including information about the perpetrator, the victim organization, and the methods of fraud employed, as well as fraud trends in general. (Respondents were not asked to identify the perpetrator or the victim.) We received 7,232 total responses to the survey, 2,690 of which were usable for purposes of this report. The data contained herein is based solely on the information provided in these 2,690 survey responses."
"2018 Report to the Nations", ACFE (2018)
OK so more than half the submitted responses were deemed unusable. That's a lot more rejects than I would normally expect for a survey which could be good, bad or indifferent: 

  • It's good if they were excluded for legitimate reasons such as being patently incomplete, inaccurate, out of scope or late - like spoiled votes in an election; 
  • It's bad (surprising and disappointing) if they were excluded illegitimately such as because they failed to support or refute some working hypothesis or prejudice;
  • It's indifferent if they were excluded for purely practical reasons e.g. they ran out of time to complete the analysis. Hopefully they used an unbiased sampling technique to trim down the data though. Perhaps the unusable responses were simply lost or corrupted for some reason.

Unfortunately, the reasons for exclusion aren't stated in the report, which to me is an unnecessary and avoidable flaw. We're reduced to guesswork. That they excluded so many responses could for instance indicate that the survey team was unusually cautious, excluding potentially as well patently dubious submissions. It could be that the survey method was changed for some reason during the survey, and the team decided to exclude those before and/or after the chosen method was used (begging further questions about what changed and how they chose the method/s).

The fact that this report comes from the ACFE strongly suggests that both the analytical methods and the team are trustworthy. Personal integrity is essential to be a professional fraud examiner, a fundamental requirement. Furthermore, they have at least disclosed the number of responses used and provide additional details in the report about the respondents. So, on balance, I'm willing to trust the report: to be clear, I do NOT think it is fraudulent! In fact, with 2,690 responses, the findings carry more weight than most vendor-sponsored "surveys" (advertisements) that I have criticised several times before.

Moving forward, I'm exploring the findings for tidbits relevant to security awareness programs, doing my level best to discount the ridiculous "infographics" they've used in the report - another unnecessary and avoidable source of bias, in my jaundiced opinion. Yes, the way metrics are reported does influence their interpretation and hence value. And no, I don't think it's necessary to resort to gaudy crayons to put key points across. Some of us aren't scared by lists, tables and graphs.

Popular posts from this blog

Pragmatic ISMS implementation guide (FREE!)

Image
Early this morning ( very  early!) I remotely attended an ISO/IEC JTC 1/SC 27/WG 1 editing meeting in London discussing the planned revision of ISO/IEC 27003:2017 . Overall, the meeting was very productive in that we got through a  long  list of expert comments on the preliminary draft standard, debated the objectives of the project and the standard and reached consensus on most points. In summary: 27003 is to be revised to align with the current 2022 releases of ISO/IEC 27001 , 27002 and 27005 : These changes are  mostly  minor aside from the new section 6.3 on ISMS changes.
Read more

Two dozen information risks that ISO forgot

Image
Selecting the wrong controls - controls that are inappropriate, ineffective, too costly, impracticable, fragile, unnecessary, counterproductive or whatever, often as a result of blind faith in fads and fashions of the day and FOMO e.g. MFA, AI, cyber Failing to select the right controls - controls that are ideal for the particular situation, both now and in perpetuity, for whatever reason - mostly ignorance and prejudice Selecting and implementing controls at the wrong time or in the wrong way (where 'wrong' includes ineffective, inappropriate, sub-optimal e.g. bolting on controls rather than designing and building them in) Inept and inaccurate identification, analysis and quantification of risk, including reliance on p oor quality (incomplete, inaccurate, out of date, misleading, unreliable ...) information about actual risks, particularly subtle and emerging risks plus those involving deliberate concealment and misdirection e.g. fraud, misinformation, disinformation, propagan
Read more

Philosophical phriday - compliance risk

Image
According to a vendor's promotional video interview I saw recently, the 'cybersecurity compliance burden' has allegedly become so significant that [customer] organisations are eagerly buying [their] software tools and services to help them manage and fulfil their obligations. The vendor's argument goes that, instead of accumulating a ragtag bunch of policies and other controls relating to user Identification and Authentication (I&A), for instance, it makes sense to:  Identify all the cybersecurity-related laws, regulations and standards that apply to the organisation; Examine them for any security control requirements relating to, say, I&A; Rationalise the I&A controls down to the smallest set that satisfies all the requirements - the lowest common denominator; Design, implement, use, manage and maintain those I&A controls; Have the I&A controls checked or audited to gain assurance that the compliance requirements are met.  OK so far? Sounds reasonab
Read more

ISMS internal audit priorities

Image
A thread on the ISO27k Forum sparked my imagination over coffee this morning. Hope had previously asked for assistance with an ISO/IEC 27001:2022 audit plan.  Bhushan offered a lengthy and generally sound response explaining how to use a spreadsheet with tabs to plan and record the audit work performed on 100% of the main body clauses and 50% of the 93 Annex A controls, day-by-day. That's OK ... except it wasn't entirely clear that he was interpreting and elaborating on the standard's actual requirements. ISO/IEC 27001 does not explicitly require, for example, that (as Bhushan stated) "ALL the management system clauses from 4 to 10 AND their sub-clauses need to be listed and audited" in an ISMS internal audit, although evidently he interprets it in that way. In clause 9.2.1, the standard states a requirement for internal audits to provide information on whether the ISMS conforms to the organization’s own requirements for the ISMS plus the requirements of the stan
Read more

Reading between the lines of ISO27001 [L O N G]

Image
ISO/IEC 27001 is a succinct, formally-worded standard for two key reasons: It is deliberately generic, being applicable to all manner of organisations regardless of difference in location/s, size, industry, maturity, structure, information risk and security status ... and so on. In effect, it specifies the lowest common denominator - the things that ALL organisations should be doing to manage their information security controls, as a minimum. The hurdle is set low enough that every organisation ought to find value in designing, implementing and operating an I nformation S ecurity M anagement S ystem as laid out in the standard. It is a certifiable standard, explicitly specifying the characteristics that every certified organisation's ISMS is expected to have. Again, it is a minimal specification with no concept of typical, average or maximum security: that is entirely down to the organisations themselves to determine, following the information risk management processes minimally de
Read more

Passionate dispassion

Image
Someone who is actively involved in, or is managing, an activity is patently not independent of it. They may well make a conscious, rational and determined effort to be objective, dispassionately reviewing evidence etc ., but their subconscious/emotional biases/prejudices and beliefs/value-systems will inevitably influence what they do. With the best will in the world, they will struggle to challenge and assess their past decisions and activities, especially if they were "certain" or "determined" or genuinely believed they were "doing the right thing". Furthermore, it is very hard for anyone to review the things they did not do, decisions they did not make or options they did not even consider. Mostly, they remain out of sight or out of the question.
Read more

45 ISO Management Systems Standards

Image
The ISO website  currently lists 45 published M anagement S ystems S tandards: 1. ISO 7101:2023 Healthcare organization management — Management systems for quality in healthcare organizations — Requirements 2. ISO 9001:2015 Quality management systems — Requirements 3. ISO 10012:2003 Measurement management systems — Requirements for measurement processes and measuring equipment 4. ISO 13485:2016 Medical devices — Quality management systems — Requirements for regulatory purposes 5. ISO 14001:2015 Environmental management systems — Requirements with guidance for use 6. ISO 14298:2021 Graphic technology — Management of security printing processes
Read more

Philosophical phriday - a noncompliance ramble

Image
In a previous philosophical phriday post , I moaned about vendors of security compliance support/management tools and services over-promising and under-delivering - an admittedly biased, even cynical opinion piece about the compliance imperative . A recent article in Corporate Compliance Insights notes that "CISOs are not just defenders against cyber threats but also champions of compliance and operational resilience". Hmmm, are CISOs 'compliance champs', really? Today, I'm discussing alternatives to being compliance-driven. How else can organisations drive their information risk, security and related concerns in a positive direction?
Read more

Adaptive SME security Crowdstrike special

Image
As if on cue, along comes a golden opportunity to consider what the Adaptive SME security  approach has to say regarding the Crowdstrike incident: That's not 20/20 hindsight but foresight: I've picked out the most relevant rows from the security controls table published in the guide 24 hours before the incident.  Although Crowdstrike primarily supplies much larger enterprises than SMEs, the incident could equally have afflicted other security software, or indeed operating systems such as Windows and assorted cloud apps commonly used by SMEs. Regardless of the details, it is a wake-up call, an opportunity to consider and respond to the information risks ... and to adapt , accordingly.
Read more