Artificial Intelligence is acting like an employee, without the rules
Artificial Intelligence is acting like an employee, without the rules
Publish Date: 2026-05-05 05:00:00
Source Domain: businessreport.co.za
- In 1979, an IBM training manual warned against letting computers make management decisions, yet today’s corporate reliance on AI-driven agents is increasingly ignoring this warning.
- The shift to agentic AI, where autonomous bots carry out complex tasks, has created an accountability gap, making organisations vulnerable to legal and financial repercussions when things go wrong.
- The “confused deputy” problem represents a heightened security threat, where AI agents’ ‘write’ access to critical systems like CRM and treasuries can be exploited, escalating into severe attacks.
- Legal systems are still struggling to adapt to the complexities of holding AI agents accountable, with cases like Air Canada’s bereavement fare incident highlighting ambiguities in accountability.
- Establishing “functional accountability” is crucial: AI agents should be supervised with human oversight, and their actions should be restricted by hard-coded limits to prevent high-stakes decisions without explicit human authorisation.
- Building a culture of scepticism towards AI agents is essential. It’s necessary to design systems where AI’s mistakes cannot lead to catastrophic management failures, adhering to the principle of Defence in Depth and leveraging a mix of people, processes, and technology for safety.
