Delusions, hallucinations and other misinformation from Chat GPT

Author: John M Dean MA CCC-SLP

Known for their ability to generate complete, full-text responses on a range of topics, Chat GPT, Bard and other “large language model” artificial intelligence (AI) tools have been dominating the news cycle for the past 6 months. Large language models (LLMs) are a new type of “generative AI”, capable of producing very convincing and comprehensive responses to a range of questions.

Although it’s tempting to use those resources to ask specific questions about issues related to your Parkinson’s, it’s becoming clear that the data coming from these tools cannot be relied upon. Because LLMs are trained on vast quantities of written text from all over the Internet, they frequently rely upon nonmedical and non-expert information sources such as Facebook and Reddit posts, among others.

In fact, a recent publication from the journal, Mayo Clinic Proceedings, identified that among the responses to 20 medical questions that were submitted to Chat GPT, reviewers uncovered 12 errors (including 5 major errors and 7 minor errors). More concerning, however, was the tendency of the responses to include fabricated citations to peer-reviewed journals. This false output is referred to as “delusions” or “hallucinations” by AI researchers. Of 59 citations reviewed, 41 were fabricated, typically by pairing factual names of authors to nonexistent articles in nonexistent journals.

While researchers and developers are working to solve what is now a known issue with false information from LLMs, it’s best to stick with reliable reference resources. That includes educational materials produced by recognized medical entities such as the Inova Parkinson and Movement Disorders Center as well as resources from national and international nonprofits. And please save specific questions about your health care needs for your doctor, nurse practitioner, or other relevant professional.

If you are not scheduled for an upcoming visit when something crops up, make sure to take advantage of My Chart to send your question directly to the team at IPMDC so your doctor or another individual on the healthcare team can provide you with a specific answer, based on your specific medical situation, privately and securely.

If you want to see the published article from Mayo Clinic proceedings yourself, you can visit their website directly. https://www.mcpdigitalhealth.org/article/S2949-7612(23)00036-6/fulltext

Leave a Reply