Tuesday 8 January 2019

Audit questions (braindump)


"What questions should an auditor ask?" is an FAQ that's tricky to answer since "It depends" is technically correct but completely unhelpful.  

To illustrate my point, here are some typical audit questions or inquiries:
  • What do you do in the area of X
  • Tell me about X
  • Show me the policies and procedures relating to X
  • Show me the documentation arising from or relating to X
  • Show me the X system from the perspectives of a user, manager and administrator
  • Who are the users, managers and admins for X
  • Who else can access or interact or change X
  • Who supports X and how good are they
  • Show me what happens if X
  • What might happen if X
  • What else might cause X
  • Who might benefit or be harmed if X
  • What else might happen, or has ever happened, after X
  • Show me how X works
  • Show me what’s broken with X
  • Show me how to break X
  • What stops X from breaking
  • Explain the controls relating to X
  • What are the most important controls relating to X, and why is that
  • Talk me through your training in X
  • Does X matter
  • In the grand scheme of things, is X important relative to, say, Y and Z
  • Is X an issue for the business, or could it be
  • Could X become an issue for the business if Y
  • Under what circumstances might X be a major problem
  • When might X be most problematic, and why
  • How big is X - how wide, how heavy, how numerous, how often ... 
  • Is X right, in your opinion
  • Is X sufficient and appropriate, in your opinion
  • What else can you tell me about X
  • Talk me through X
  • Pretend I am clueless: how would you explain X
  • What causes X
  • What are the drivers for X
  • What are the objectives and constraints relating to X
  • What are the obligations, requirements and goals for X
  • What should or must X not do
  • What has X achieved to date
  • What could or should X have achieved to date
  • What led to the situation involving X
  • What’s the best/worst thing about X
  • What’s the most/least successful or effective thing within, about or without X
  • Walk or talk me through the information/business risks relating to X
  • What are X’s strengths and weaknesses, opportunities and threats
  • What are the most concerning vulnerabilities in X
  • Who or what might threaten X
  • How many changes have been made in X
  • Why and how is X changed
  • What is the most important thing about X
  • What is the most valuable information in X
  • What is the most voluminous information in X
  • How accurate is X …
  • How complete is X …
  • How up-to-date is X …
    • … and how do you know that (show me)
  • Under exceptional or emergency conditions, what are the workarounds for X
  • Over the past X months/years, how many Ys have happened … how and why
  • If X was compromised in some way, or failed, or didn’t perform as expected etc., what would/might happen
  • Who might benefit from or be harmed by X 
  • What has happened in the past when X failed, or didn’t perform as expected etc.
  • Why hasn’t X been addressed already
  • Why didn’t previous efforts fix X
  • Why does X keep coming up
  • What might be done to improve X
  • What have you personally tried to address X
  • What about your team, department or business unit: what have they done about X
  • If you were the Chief Exec, Managing Director or god, what would you do about X
  • Have there been any incidents caused by or involving X and how serious were they
  • What was done in response – what changed and why
  • Who was involved in the incidents
  • Who knew about the incidents
  • How would we cope without X
  • If X was to be replaced, what would be on your wishlist for the replacement
  • Who designed/built/tested/approved/owns X
  • What is X made of: what are the components, platforms, prerequisites etc.
  • What versions of X are in use
  • Show me the configuration parameters for X
  • Show me the logs, alarms and alerts for X
  • What does X depend on
  • What depends on X
  • If X was preceded by W or followed by Y, what would happen to Z
  • Who told you to do ... and why do you think they did that
  • How could X be done more efficiently/effectively
  • What would be the likely or possible consequences of X
  • What would happen if X wasn’t done at all, or not properly
  • Can I have a read-only account on system X to conduct some enquiries
  • Can I have a full-access account on test system X to do some audit tests
  • Can I see your test plans, cases, data and  results
  • Can someone please restore the X backup from last Tuesday 
  • Please retrieve tape X from the store, show me the label and lend me a test system on which I can explore the data content
  • If X was so inclined, how could he/she cause chaos, or benefit from his/her access, or commit fraud/theft, or otherwise exploit things
  • If someone was utterly determined to exploit, compromise or harm X, highly capable and well resourced, what might happen, and how might we prevent them succeeding
  • If someone did exploit X, how might they cover their tracks and hide their shenanigans
  • If X had been exploited, how would we find out about it
  • How can you prove to me that X is working properly
  • Would you say X is top quality or perfect, and if not why not
  • What else is relevant to X
  • What has happened recently in X
  • What else is going on now in X
  • What are you thinking about or planning for the mid to long term in relation to X
  • How could X be linked or integrated with other things
  • Are there any other business processes, links, network connections, data sources etc. relating to X
  • Who else should I contact about X
  • Who else ought to know about the issues with X
  • A moment ago you/someone else told me about X: so what about Y
  • I heard a rumour that Y might be a concern: what can you tell me about Y
  • If you were me, what aspects of X would concern you the most
  • If you were me, what else would you ask, explore or conclude about X
  • What is odd or stands out about X
  • Is X good practice
  • What is it about X that makes you most uncomfortable
  • What is it about this audit that makes you most uncomfortable
  • What is it about me that makes you most uncomfortable
  • What is it about this situation that makes you most uncomfortable
  • What is it about you that makes me most uncomfortable
  • Is there anything else you’d like to say
I could go on all day but that is more than enough already and I really ought to be earning a crust! If I had more time, stronger coffee and thought it would help, I might try sorting and structuring that braindump ... but in many ways it would be better still if you did so, considering and revising the list to suit your purposes if you are planning an audit. 

Alternatively, think about the questions you should avoid or not ask. Are there any difficult areas? What does that tell you?

It's one of those situations where the journey trumps the destination. Developing a set of audit concerns and questions is a creative process. It's fun.

I’m deliberately not specifying “X” because that is the vital context. The best way I know of determining X and the nature of the questions/enquiries arising is risk analysis. The auditor looks at the subject area, considers the possibilities, evaluates the risks and picks out the ones that are of most concern, does the research and fieldwork, examines the findings … and re-evaluates the situation (possibly leading to further investigation – it’s an iterative process, hence all the wiggly arrows and loops on the process diagram). 

Auditing is not simply a case of picking up and completing a questionnaire or checklist, although that might be part of the audit preparation. Competent, experienced auditors feed on lists, books, standards and Google as inputs and thought-provokers for the audit work, not definitive or restrictive descriptions of what to do. On top of all that, the stuff they discover often prompts or leads to further enquiries, sometimes revealing additional issues or risks or concerns almost by accident. The real trick to auditing is to go in with eyes, ears and minds wide open – curious, observant, naïve, doubtful (perhaps even cynical) yet willing to consider and maybe be persuaded.

No comments:

Post a Comment

The floor is yours ...