AI finds drug safety signals in clinical notes

Source Domain: news.vumc.org

Here is an unordered list summarizing the key points from the article:

Investigative Approach Using LLMs: Large Language Models (LLMs) have been studied to detect immune-related adverse events (irAEs) associated with immune checkpoint inhibitors (ICIs) from clinical notes and electronic health records (EHR).
Zero-Shot Learning Utilization: The study employs zero-shot learning, whereby LLMs, like GPT-3.5, GPT-4, and the best-performing GPT-4o, are tested using only a detailed prompt without any prior examples to identify irAEs in clinical notes.
Testing on Multiple Datasets: The models were evaluated on clinical notes from Vanderbilt Health, University of California at San Francisco, and Roche trials. Each dataset was significant in contributing to the study’s findings.
Performance Metrics: F1 scores were the primary measure of the models’ performance. While promising, average scores across the tested models (ranging between 56% to 66%) indicated the need for further improvement to be useful for clinical decisions.
Potential Benefits: The researchers argue that although the current performance of LLMs does not yet meet the clinical decision support standards, it could facilitate quicker and more economical detection of irAEs, aiding precision medicine initiatives.
Study Implications: The findings highlight the potential of advanced AI techniques like zero-shot learning in improving the efficiency and speed of monitoring drug safety signals in cancer immunotherapies.
Interdisciplinary Collaboration: The study included experts across biomedical informatics, oncology, and data science from Vanderbilt University, including Yaomin Xu, Eric Mukherjee, and Justin Balko, among others.
Related Research: A separate study from December in JAMA Oncology, involving the same team, confirmed an increased risk of severe skin reactions (SJS/TEN) associated with ICI treatment, providing further insights into the adverse effects of these treatments.