Blog

Comment: AI and accountability – who is responsible?

Blog Page Header Template (16)

Read time: 4 minutes
Summary: Dr. Valentina Dotto, PhD, Senior Policy Adviser at the Chartered Governance Institute, draws on CGI's latest research and recent discussions between major technology companies and the US government to explore the growing governance challenges surrounding AI oversight, board capability and organisational responsibility.

AI has become a prominent topic in public discourse, but we are now seeing high-profile discussions specifically about the governance of these technologies. Last week, US President Donald Trump met with the leading Artificial Intelligence companies, including OpenAI, Anthropic, Nvidia, SpaceX, Meta and Google, at a ‘Super Intelligence’ summit to address the potential risks.  

While the summit was noticeably light on specific details of what safeguarding and governance guardrails will look like, the conclusion is the signing of a "morally binding" document stating that these companies are responsible for ensuring the safety of their own products. But is self-regulation enough? This outcome is contrary to recommendations arising from CGI’s latest research: AI needs distinct parameters regarding accountability and governance oversight.   

As part of the Super Summit, Trump conceded that AI companies would be assessed by "independent auditors" to ensure their systems are working correctly and to mitigate risk. Our own research shows that rather than replacing governance expertise, AI appears to be increasing the need for contextual judgement and organisational understanding.  

‘Equipping Governance Professionals to Lead AI Conversations’ uses cross-sector evidence to highlight the areas where governance is falling out of step with advancing technology. The findings show that governance structures aren't adapting at the same pace as AI adoption, exposing a significant risk. 

Research participants frequently described a shift away from routine administration and towards activities centred on interpretation, challenge and assurance. As AI systems become more capable of processing information, the governance professional's role involves helping boards make sense of that information within the context of the organisation's strategy, culture and risk profile.

Between February and May 2026, CGI conducted research involving governance professionals from the corporate, public and charity sectors, including roundtables and survey responses from more than 160 participants. Wefound that while AI is often discussed as a technology issue, it is much more significant than that.
AI is being widely used for organisational output, yet legal responsibility still sits in the hands of humans, creating a disconnect between where accountability lies legally and operationally. We examined the risks around accountability, capability, and control to discover the key questions governance professionals should start asking. Notably, do you have visibility of where and how AI is actually being used across the organisation? Do directors have sufficient practical competence to interrogate AI outputs and challenge assumptions? And, can you evidence that AI-assisted decisions are subject to appropriate human oversight and challenge? 

The findings show that many boards are accountable for systems that they cannot yet fully see, describe or oversee. With such rapid change, the question is whether boards will be able to assess both the opportunities and risks. The recent hacking of the Australian universal health insurance platform by an OpenAI agent reinforces just how damaging the potential risks can be.  

Similarly, most survey respondents reported that shadow AI usage occurs at least occasionally, indicating that AI adoption is happening outside formal governance structures, including by board members themselves. A recent report by the Diligent Institute found that 66% of board members admitted to uploading board packs into open-source software like ChatGPT. From drafting, summarisation, meeting support to information synthesis, usage appears to be driven from the bottom up, raising a broader question about how organisations are capturing and learning from these emerging practices, and how those insights inform wider organisational approaches to AI. 

One of the most significant issues that came to light through these discussions concerns records and evidence. Governance professionals consistently highlighted tensions around AI-generated transcripts, meeting recordings and automated summaries. To this extent, the Institute has been developing a new guidance note, “Digital Boardroom Governance: Board Meeting Hygiene in the Digital and AI-Enabled Boardroom” which we intend to publish before year-end. These tools offer efficiency, but they also raise debates about discoverability, retention, record integrity and the distinction between the official record and machine-generated artefacts. How will organisational judgement be evidenced and defended under scrutiny going forward? 

For example, 50% of respondents shared that they have a human-in-the-loop protocol for auditing AI-generated content. Human supervision is more commonly applied after system-generated outputs are produced, rather than during decision-making. Our research shows that participants consistently rejected the idea that AI reduces accountability. But with all that has come to light, is this process rigorous enough? Especially considering that directors remain responsible for exercising reasonable care, skill and diligence, regardless of the tools used to support governance decision-making.  

Despite this, just under half of governance professionals and boards admitted to having received any formal AI training in the past 12 months. As AI plays a more accepted part in organisational processes, boards need to be confident in challenging outputs, understand limitations and seeking appropriate assurance. More than half of governance professionals said they now feel that AI literacy should become a mandatory component of director induction and ongoing professional development. 

Most organisations have developed awareness of AI, but few have developed the practical competence required to oversee its use effectively in real governance environments. Recent concerns over advanced AI systems exhibiting unexpected and potentially deceptive behaviours have reinforced the importance of discussions around AI safety, transparency, oversight and accountability. For governance teams, this means considering not only organisational risk, but also the broader ecosystems on which organisations depend. 

Ultimately, the conversations around accountability should not be confined to governments and technology companies. The goal is to ensure governance frameworks evolve with AI, so that the benefits can be utilised whilst mitigating the risk. Boards are not deciding whether to engage with AI, but whether they are equipped to shape that engagement deliberately, transparently and credibly. 

Read the full CGI report: Equipping Governance Professionals to Lead AI Conversations.

Find out more about our AI Governance Series - a comprehensive programme designed for governance professionals, company secretaries, board members, and senior leaders seeking to confidently navigate the opportunities, risks, and responsibilities associated with Artificial Intelligence.