I hope I didn’t scare you with my blog last week about confirmation bias and the increased risk to our decision making from our here-to-please and oh-so-convincing AI App team member. Even so, this risk poses a significant governance challenge for all organisations: ensuring accountability for decisions made.

Across the organisations I have been working with this year, accountability for decisions made by AI, by individuals using AI, or by AI overseen by a human in the loop has been built into their governance approach for AI. Great. But let’s go deeper.

First, we have “garbage in garbage out”. We’re being trained in, and encouraged to use, sound AI prompts and organisations are also (finally) focussing on Data Governance.  However, all that only goes so far. Some of the information AI uses to form its opinions has errors and can easily be taken out of context.

The bigger question is how did the accountable person challenge themselves or the decision made by AI? Was the governance around the decision good enough? And what is good enough in any one particular situation?

Let’s start by setting some expectations on important decisions or those with significant potential for adverse impacts on stakeholders. Did the accountable person:

• Ask for and look for and test assumptions?
• Interrogate the source data and assess the context in which it was created?
• Look for contrary evidence?
• Understand the limits of AI in providing an answer?

If you are a board member, your responsibilities are broader now you know the extent AI is being used in decision making. You can’t simply assume there will be a capable ‘human in the loop’. That is just a dreaded box ticking compliance exercise!