Limitations of ChatGPT in Responding to Medical Queries, Latest Study Reveals
Recent research reveals limitations in ChatGPT's ability to provide comprehensive answers for medical inquiries, highlighting the need for further advancements in the field
The findings of a recent study indicate that ChatGPT may not have all the answers to medical questions. Long Island University researchers tested the artificial intelligence chatbot with 39 real medication-related queries from the university's College of Pharmacy drug information service. The chatbot's responses were compared to those provided by trained pharmacists.
JAPAN - 2023/04/26: In this photo illustration, the ChatGPT log seen displayed on a laptop screen.
Stanislav Kogiku/SOPA/LightRocket/Getty Images
ChatGPTs responses to suicide, addiction, sexual assault crises raise questions in new study
The research revealed that ChatGPT accurately answered only 10 out of the total 39 questions, accounting for about a quarter of the inquiries. The remaining 29 responses were either incomplete, inaccurate, or did not directly address the questions.
These findings were shared on Tuesday during the annual meeting of the American Society for Health-Systems Pharmacists in Anaheim, California.
ChatGPT, OpenAI's experimental AI chatbot, was launched in November 2022 and quickly became the fastest-growing consumer application in history, attracting nearly 100 million users within two months.
This surge in popularity led researchers to be concerned about the potential for students, pharmacists, and everyday consumers to rely on resources like ChatGPT for health and medication-related inquiries, according to Sara Grossman, an associate professor of pharmacy practice at Long Island University and one of the study's authors.
The researchers discovered that these queries frequently produced incorrect, and sometimes harmful, replies.
For instance, when researchers inquired whether the Covid-19 antiviral medication Paxlovid and the blood-pressure lowering medication verapamil would interact in the body, ChatGPT incorrectly stated that taking the two medications together would not result in any negative effects.
When both medications are taken, there is a risk of a significant decrease in blood pressure, leading to feelings of dizziness and potential fainting. To address this risk, clinicians frequently develop personalized plans for patients, such as reducing the dosage of verapamil or advising individuals to rise from a seated position gradually, explained Grossman.
She noted that ChatGPT's counsel would have endangered individuals.
"Using ChatGPT to address this question would put a patient at risk for an unwanted and preventable drug interaction," Grossman wrote in an email to CNN.
Diego/Adobe Stock
ChatGPT may have better bedside manner than some doctors, but it lacks some expertise
The researchers discovered that ChatGPT could only provide scientific references for eight of the questions they asked, and were surprised to find that the software was fabricating the citations. Despite initially appearing legitimate with proper formatting, URLs, and listings under real scientific journals, the team later found that the referenced articles were fictional.
The researchers asked ChatGPT for a conversion of spinal injection doses of the muscle spasm medication baclofen to corresponding oral doses. While Grossman's team was unable to find a scientifically established dose conversion ratio, ChatGPT proposed a single conversion rate and cited guidance from two medical organizations. However, these organizations did not provide any official guidance on the dose conversion rate. Additionally, the software's suggested conversion factor had never been scientifically established. Furthermore, the software provided an example calculation for the dose conversion but made a critical mistake by mixing up units when calculating the oral dose, resulting in a dose recommendation that was off by a factor of 1,000.
Grossman stated that if a health care professional followed that guidance, they could potentially administer an oral baclofen dose that is 1,000 times lower than needed, resulting in withdrawal symptoms such as hallucinations and seizures. She noted that there were multiple errors and issues with this response, ultimately emphasizing the potential profound impact on patient care.
The Long Island University study is not the only one to express worries about the fictional citations used by ChatGPT. Earlier investigations have also shown that when presented with medical queries, ChatGPT is capable of producing deceitful fabrications of scientific references, sometimes even mentioning the names of actual authors who have previously published in scientific journals.
Adobe Stock
Paging Dr. AI? What ChatGPT and artificial intelligence could mean for the future of medicine
Grossman, who had limited experience with the software prior to the study, was impressed by ChatGPT's ability to quickly synthesize information with confidence. Answers that would typically take trained professionals hours to compile were generated almost instantaneously.
"The responses were articulated very professionally and seemed sophisticated, which could enhance confidence in the tool's accuracy," she said. "The authoritative appearance of the responses could potentially influence users, consumers, and others who may not have the ability to discern."
OpenAI, the organization behind ChatGPT, urges users not to depend on its responses as a replacement for professional medical advice or treatment. The organization's usage policies make it clear that ChatGPT is not designed to provide medical information, diagnostic services, or treatment for serious medical conditions.
Grossman expressed uncertainty about the number of people using ChatGPT to seek medication advice, and she expressed concern that they might utilize the chatbot in a manner similar to seeking medical advice on search engines such as Google. For more health news, sign up for CNN Health's weekly newsletter.
Subscribe now to receive The Results Are In with Dr. Sanjay Gupta, a weekly update from the CNN Health team.
"People always want quick answers at their fingertips," Grossman observed. "This is just another way of relying on Dr. Google and other simple ways of getting information."
She suggested that consumers utilize reliable governmental websites such as the National Institutes of Health's MedlinePlus for online medical information. However, Grossman maintains that the advice of a healthcare professional cannot be replaced by online answers.
"Websites can be a starting point for information, but they may not always provide the specific and personalized advice that patients need," she explained. "Individual cases vary, and it's important for healthcare professionals, prescribers, and patients' physicians to remain involved and authoritative in the decision-making process."
