Medical AI transcriber for Ontario doctors ‘hallucinated,’ generated errors: auditor general

Source Domain: www.cbc.ca

Ontario’s Auditor General Shelley Spence highlighted that AI note-taking tools intended for use by doctors exhibited serious mistakes and incomplete information, including inaccuracies in medical notes, potentially impacting patient health.
The provincial auditor conducted tests on 20 AI scribe systems which revealed significant inaccuracies, including “hallucinations” where systems fabricated suggestions or information not presented by doctors.
Nine AI systems were found to fabricate information, such as recommending therapy or ordering unnecessary tests, while 12 contained incorrect drug prescriptions and 17 missed details about mental health issues.
Fourteen AI programs didn’t submit required third-party audits or certifications during the procurement process, although four were still approved.
The government issued guidelines for doctors to manually review AI-generated notes to catch errors, and the auditor recommended enforcing doctor attestations for reviewed notes.
The auditor emphasized the importance of thorough testing for AI systems to ensure reliability and safety, while a minister defended the tools, claiming oversight by qualified professionals and that they help save doctors’ time.
Despite approximately 5,000 physicians using AI scribes with no known reports of harm, caution was urged for proper oversight, testing, and security measures within AI tool deployment.

You may have missed