ChatGPT May Help Patients Better Understand Cirrhosis, Liver Cancer

Generative artificial intelligence (AI) chatbot ChatGPT may be helpful for delivering some medical information to patients with cirrhosis or hepatocellular carcinoma (HCC), according to an analysis of AI-generated answers to patient questions.

The analysis of 164 questions showed that ChatGPT provided correct answers on the majority of questions about cirrhosis (79.1%) and HCC (74%) based on independent grades by two transplant hepatologists and one reviewer, according to Brennan Spiegel, MD, of Cedars-Sinai Medical Center in Los Angeles, and colleagues.

However, a smaller proportion of answers provided by ChatGPT rose to the level of "comprehensive" -- the highest of four grades given by the reviewers -- at 47.3% in cirrhosis and 41.1% in HCC, the team reported in Clinical and Molecular Hepatology.

Its performance was better in the domains of basic knowledge, lifestyle, and treatment than in diagnosis and prevention, the researchers noted.

Co-author Yee Hui Yeo, MD, also of Cedars-Sinai, told MedPage Today that ChatGPT provided substantial answers to frequently asked questions about these diagnoses, but was still limited in providing specific information such as treatment durations.

He said ChatGPT performed well regarding questions about coping with the emotional and mental burden of cirrhosis and HCC, which the researchers suggested could mean the AI chatbot is "able to provide empathetic and practical advice to patients and caregivers."

For instance, ChatGPT "acknowledged the patient's likely emotional response to their diagnosis," and it "provided clear and actionable starting points for a patient who had just been diagnosed with HCC," the researchers reported.

Yeo said using ChatGPT in this approach could eventually help extend a healthcare professional's ability to share their knowledge with patients in a manner that also saves the provider time.

"I think it's going to reduce our workflow because many patients have many questions," Yeo said, adding that most patients have a similar set of questions they want answered.

"You will find that actually there are some questions that [patients are] really interested in and curious about -- about the disease, about the medication they use -- so when it comes to either similar or overlapped questions, these questions can be produced by ChatGPT [and] then verified by the doctors," he said. "It will save a lot of time."

In addition to generating answers to frequently asked questions about a patient's specific diagnosis, a key strength of ChatGPT is its ability to simplify complex ideas into conversational text, Yeo said.

"We think that it could explain [a diagnosis] in a way that is better than many of the hepatologist or family medicine [providers] ... in a way that is very reader friendly," he said.

The researchers also emphasized that ChatGPT is free to the public, which makes it more accessible to patients.

Of the 164 questions the investigators used as prompts for ChatGPT, 91 were frequently asked questions about cirrhosis and 73 were frequently asked questions about HCC. The team gathered the questions from professional societies and institutions as well as a patient support group on Facebook.

ChatGPT's answers were then graded using a 4-point system that ranged from "comprehensive" as the highest grade to "completely incorrect" as the lowest. The questions were then divided into five categories -- basic knowledge, diagnosis, treatment, lifestyle, and preventive medicine.

Yeo emphasized that the results were produced using the December 15 version of ChatGPT, based on the GPT-3.5 model. OpenAI has since released GPT-4, which has proven to be a vast improvement from the earlier version. Yeo said the team has already been studying the performance of the latest version and expects to publish those results in the coming months.

Comment