Healthwatch England has raised significant concerns regarding the use of AI technology, specifically AI scribes, in healthcare settings. These systems, designed to listen to and transcribe patient consultations, have been found to potentially misrecord critical medical information, leading to misconceptions about diagnoses and treatments.
One serious incident involved a woman who was alarmed to find that the AI tool misreported her medical status as having “demyelination,” a condition associated with severe nerve injury that can precede multiple sclerosis. It wasn’t until this NHS health professional challenged the accuracy of the AI-generated summary of her MRI results that the error was rectified, revealing the correct report indicated “null demyelination.” This experience, she noted, was distressing, highlighting the potential psychological implications of inaccuracies in medical records.
Healthwatch England’s investigation found similar instances where patients identified errors that had gone unnoticed by healthcare professionals. In one case, the AI confused the name of a prescribed medication, which was only caught by the patient. In another scenario, a summary letter failed to inform a patient that their consultant had recommended a prescription renewal, which could have hindered their access to necessary medication.
With the NHS rapidly adopting AI technology, there are warnings that these mistakes could lead to serious patient safety issues if they remain uncorrected in medical records, ultimately affecting patient care.
The government’s 10-year health plan for the NHS aims to use AI to alleviate the administrative burden faced by staff, allowing them to prioritize patient care. However, Healthwatch has emphasized the necessity for robust processes that allow patients to report and rectify mistakes stemming from AI use.
Rachel Power, chief executive of the Patients Association, noted that trust in such technologies hinges on effective communication and genuine collaboration with patients, both of which are currently lacking. The absence of regulatory measures from the Medicines and Healthcare products Regulatory Agency to classify AI scribes as medical devices raises additional concerns, with fears about oversight and safety unchecked across England’s healthcare system.
Across England, healthcare professionals are reportedly utilizing 27 different AI scribing tools. There’s increasing concern that medical practitioners could face liability for errors arising from these AI systems, raising questions about accountability within the healthcare sector.
Recent criticisms have emerged regarding an AI receptionist in Rotherham, which reportedly struggled to understand patients with strong Yorkshire accents. Such issues underscore the variability of AI performance depending on context, prompting caution among health professionals as AI is integrated more into clinical workflows.
Dr. Shier Ziser Dawood, a GP based in London, illustrated the issues surrounding AI’s reliability by recounting an incident where an AI scribe incorrectly documented that she prescribed a patient Prozac, which had never been discussed. This points to a phenomenon known as “hallucinations,” where AI inaccurately generates information that was not part of the consultation. The need for physicians to meticulously review AI outputs for accuracy detracts from the intended efficiency gains, potentially imposing additional burdens on healthcare providers.
Dr. Charlotte Blease from Uppsala University highlighted that GPs using voice technology in consultations often find that mistakes tend to escalate when dealing with multiple patients or complex medical histories, particularly for those whose primary language is not English. Nevertheless, she acknowledged that human errors are an ongoing reality in healthcare, suggesting that AI may not necessarily lower the overall error rate.
Interestingly, over half of the UK GPs surveyed believe that the AI-generated medical records are more accurate than their own notes, possibly indicating a shifting perception of technology within clinical practice.
Editor’s Take
The integration of AI in healthcare is a double-edged sword. While these technologies can enhance efficiency, their current shortcomings raise serious concerns about patient safety and accuracy. For users and healthcare providers alike, the implications of these errors are profound. As AI systems continue to evolve, establishing stringent oversight and clearer communication pathways will be essential to maintain trust in their use.
Source: www.theguardian.com
