ChatGPT for advice on common GI endoscopic procedures: the promise and the peril.

We queried ChatGPT on common types of endoscopic procedures, including upper GI endoscopy (EGD), colonoscopy, ERCP, and EUS. Queries directed to ChatGPT related to the nature of the procedure, alternatives, and risks. We specifically asked for the quantum of each potential risk. We further queried ChatGPT on whether preparation for GI endoscopy was required, whether medication adjustments were required, and whether follow-up was necessary. Three iterations were performed for each set of questions to qualitatively examine for run-to-run variation, with a fresh ChatGPT session initiated for each iteration. Three authors (D.Y.Z.L., G.G.R.S., and J.Y.M.T.) collaboratively identified key elements of high-quality periprocedural counseling for endoscopy: factual accuracy, comprehensiveness of advice, presence or absence of major safety issues, internal consistency of the response, and overall linguistic fluency. These criteria were used to qualitatively evaluate the responses of ChatGPT. The individual observations of each researcher were aggregated in the final analysis.

The full questions and response logs are recorded in Appendices 1 , 2 , and 3 (available online at www.igiejournal.org ). We summarized the important strengths and limitations of the responses, paying special attention to whether there was factually incorrect information, potentially misleading information, or important omissions. These are also qualitatively summarized in Table 1 and elaborated on below. Table 1 Qualitative appraisal of advice Procedure Type of information Positive aspects Negative aspects (minor) Negative aspects (major) All Periprocedural management Correct advice on fasting, bowel preparation, stopping antithrombotics, postsedation care Advises patient to seek further clarification from healthcare provider Correct advice to stop iron supplements and herbal medications, but inconsistently Potentially misleading advice to continue “diabetes and heart disease” medications in 1 iteration Colonoscopy Nature of procedure Correct description of colonoscopy Indications Correct indications of screening for colorectal cancer, follow-up of abnormal results, follow-up of known pathology (colorectal cancer, polyps) Correct indication of follow-up of inflammatory bowel disease, but stated only in 1 iteration Alternatives Correct alternatives of fecal occult blood test, fecal immunohistochemistry test, stool DNA test, and CT colonography Alternatives described pertain mainly to screening and are inconsistent with evaluation of symptoms Flexible sigmoidoscopy correctly offered but only in 2 iterations Barium enema is obsolete but was offered in 2 iterations Risks Risks of bleeding, pain, infection, adverse reaction to sedation, discomfort correctly stated, with correct quantum of risk Risk of false-negative results described in only 1 iteration Follow-up Correctly describes that need for repeat colonoscopies depends on indications (eg, 10 y for screening colonoscopy with no abnormalities found) EGD Nature of procedure Correct description of EGD Indications Correct indications such as evaluation of symptoms or bleeding, assessing previous treatment Alternatives Described barium swallow as a potential alternative to EGD Axial imaging (CT or magnetic resonance imaging) offered in 2 iterations Outdated or less-relevant alternatives offered, such as upper GI series, hydrogen breath test, gastric-emptying study, pH monitoring, esophageal manometry Stool test for blood offered in 1 iteration “Upper GI endoscopy” offered in 1 iteration (ChatGPT described this as a “similar procedure to EGD, but only examines the esophagus and stomach rather than the entire digestive tract”) Risks Risks of bleeding, pain, infection, adverse reaction to sedation, discomfort correctly stated, with correct quantum of risk One iteration advised a risk of false-negative results Another iteration advised a risk of false-negative and false-positive results Follow-up Correctly mentioned that the need for repeat EGD depends on original indication and subsequent result One iteration mentioned that EGD might be part of a regular screening program ERCP Nature of procedure Correct description of ERCP Indications Correct indications described including bile duct stones, diagnosis of biliary and pancreatic tumors Stent placement of biliary system mentioned in 1 iteration Incorrect indications of investigation of abdominal pain, jaundice, abnormal liver function tests, pancreatitis, sphincter of Oddi dysfunction Alternatives None described entirely correctly MRCP, CT, and EUS described as diagnostic alternatives in 2 iterations Incorrect alternatives of cholangiography and laparoscopic surgery described Magnetic resonance imaging and MRCP described as different diagnostic modalities Risks Risks of bleeding, pain, perforation, pancreatitis, and adverse reactions to sedation correctly stated, with correct quantum of risk Risk of infection mildly underestimated (described as low or 1%-2%, when it can be up to 3%) Risk of contrast adverse events was overestimated (described as low or 1%-3%, when in fact it is negligible) No iteration mentioned minor risk related to fluoroscopic x-ray exposure Follow-up Correctly mentioned that the need for repeat ERCP depends on original indication and subsequent result No iteration mentioned the potential need for repeat endoscopy to remove a stent if it was placed EUS Nature of procedure Correct description of EUS Correct information provided (unprompted) that fine-needle biopsy sampling/FNA may be done with EUS Indications Correct description of common indications, such as diagnosis of mural lesions, pancreatic and biliary lesions, staging lymph nodes for malignancy, and facilitating tissue biopsy sampling Small number of incorrect indications, such as diagnosis of unexplained abdominal pain, weight loss, chronic diarrhea, and GI bleeding No mention of celiac plexus interventions, although this is a less-common use of EUS Alternatives Correct description of potential alternatives such as imaging (CT, magnetic resonance imaging) and other endoscopic modalities (standard endoscopy, ERCP) Transabdominal US mentioned as a valid alternative in only 1 iteration Positron emission tomography mentioned as an alternative in 2 iterations; although this may be a valid imaging option, it is usually not a first-line alternative Risks Risks of bleeding, pain, perforation, and adverse reactions to sedation correctly stated, with correct quantum of risk Appropriate mention of risk of pancreatitis if EUS is used as part of a pancreas intervention Appropriate mention of increased bleeding risk with biopsy sampling and interventional procedures No mention of the risk of malignant seeding when EUS combined with fine-needle biopsy sampling/FNA, although the quantum of this risk is low and only described in case reports One iteration overestimated perforation risk (described as 1%-3%, when it is in fact <1% and equivalent to standard endoscopy when EUS alone is performed) Risk of false-negative results not described when unprompted, although correct advice was given when specifically asked about the possibility of false negatives One iteration incorrectly stated that a mitigating factor for infection is that many parts of the EUS are single-use and disposable Follow-up Correctly mentioned that the need for repeat EUS depends on original indication and subsequent result Qualitative appraisal of advice Overall, ChatGPT was able to answer the questions clearly and in plain English. Although phrasing differed between iterations, the content remained generally consistent. There was minimal use of healthcare jargon, and most technical words had accompanying nonexpert transliterations. Most responses were accompanied by advice to discuss the specific options or recommendations with a doctor. With regard to periprocedural management, ChatGPT was able to consistently give advice on appropriate fasting and bowel preparation. It also provided unprompted advice on postsedation care, such as arranging for transport home after the procedure. Important postprocedural advice, such as rest, hydration, and monitoring for pain or bleeding symptoms, was consistently applied. Advice on medications to stop preprocedurally was mostly correct, with universal advice to stop antithrombotic drugs. Advice on stopping iron supplements and herbal medications was included in most, but not all, responses. However, information on continuation of chronic medications was very inconsistent, with most iterations failing to alert patients to stop or adjust diabetes medications while fasting. One iteration advised continuation of “diabetes and heart disease” medications, which could have been incorrectly interpreted as advice to continue antithrombotics and/or unadjusted diabetes medication while fasting. For colonoscopy and EGD, procedures were correctly described. Well-accepted indications such as symptom investigation, follow-up of abnormal test results, and screening (for colonoscopy) were disclosed. Advice on alternatives for colonoscopy mainly related to screening colonoscopy, with alternatives like fecal occult blood test, CT colonography, and flexible sigmoidoscopy, was presented. One iteration presented double-contrast barium enema as an alternative to colonoscopy. However, advice on alternatives for EGD contained significant limitations, such as inappropriate alternatives (stool tests for blood, blood tests to evaluate for anemia) and second-line investigations (hydrogen breath tests, esophageal manometry, gastric-emptying studies). Important risks such as bleeding, perforation, infection, discomfort, adverse reactions to sedation, and false-negative results were conveyed unprompted. The quantum of risks was conveyed qualitatively as “low” for most adverse events, with some iterations providing quantitative low ranges of risk from .1% to 1%. These conformed to generally quoted risks for diagnostic colonoscopy 6 and EGD. 7 Most answers were appropriately qualified with various risk modifiers; for example, biopsy sampling and polypectomy were quoted as potential risk-increasing factors for bleeding in colonoscopy. Advice on the need for repeat colonoscopy was accurate and thorough and included qualifiers on different screening intervals for patients with polyps and inflammatory bowel disease. Advice on repeating EGD was also sound, with descriptions on need for repeat endoscopy in cases of Barrett’s esophagus and to obtain more histology if required. However, 1 iteration mentioned EGD as part of a regular screening program, a practice that is not recommended by most major professional societies. EUS was generally correctly described, with well-accepted diagnostic indications such as diagnosis of mural lesions, pancreatic and biliary lesions, staging lymph nodes for malignancy, and facilitating tissue biopsy sampling given. Although unprompted, information about the potential for EUS to guide procedures such as endoscopic drainage was also supplied. Some incorrect indications, such as diagnosis of unexplained abdominal pain, weight loss, chronic diarrhea, and GI bleeding, were supplied. There was no mention of celiac plexus interventions as an indication, although this is a less common use of EUS. Advice on alternatives generally pertained to diagnostic uses of EUS, with potential alternatives such as imaging (CT, magnetic resonance imaging) and other endoscopic modalities (standard endoscopy, ERCP) stated. However, positon emission tomography was listed as a potential alternative, although it is generally not an appropriate first-line alternative. Important risks such as bleeding, perforation, infection, discomfort, and adverse reactions to sedation were conveyed unprompted. The quantum of risks was conveyed qualitatively as “low” for most adverse events, with some iterations providing quantitative low ranges of risk. One iteration overestimated perforation risk at 1% to 3%, (when it is in fact <1% and equivalent to standard endoscopy when EUS alone is performed 8 ). Otherwise, risk estimates were correct, with appropriate qualification about increased risk of adverse events under specific situation (eg, increased risk of bleeding when biopsy sampling is also performed). False-negative results were not listed as a potential risk when unprompted, although correct information about the risk of false negatives was given when ChatGPT was specifically asked. Advice on the need for a repeat EUS was generally sound, with credible reasons such as follow-up of an initial inconclusive result, repeat with a view to biopsy sampling if it was not originally performed, and surveillance of lesions such as pancreatic cysts. Although advice on EGD and colonoscopy was generally safe and credible, advice on ERCP contained numerous misleading and potentially dangerous statements. Most importantly, indications given for ERCP seemed to focus on the diagnostic qualities of ERCP, when in fact the modern use of ERCP is mainly therapy (eg, removal of bile duct stones, treatment of biliary strictures). Outdated or unacceptable indications for ERCP were varyingly described, such as suspected sphincter of Oddi dysfunction and evaluation of unexplained abdominal pain. Similarly, alternatives included outdated or unacceptable modalities, such as percutaneous transhepatic cholangiogram and laparoscopic surgery as diagnostic procedures. Major risks of ERCP were correctly described with some limitations. Major risks such as bleeding, pain, perforation, pancreatitis, and adverse reactions to sedation were correctly described. However, the risk of infection was mildly underestimated (described as “low” or 1%-2% when it can be up to 3%), and the risk of contrast adverse events was overestimated (described as “low” or 1%-3% when in fact it is negligible). 9 None of the iterations informed the patient about the minor risk related to fluoroscopic x-ray exposure.

We found that advice dispensed by ChatGPT was easily comprehended and would be accessible to a layperson. Information pertaining to periprocedural care and EGD, colonoscopy, and EUS (mainly for diagnostic indications) was of fair quality, with minor inconsistencies. However, information on ERCP had significant inaccuracies and errors, which varied across the different iterations. We surmise that some of the limitations stem from ChatGPT’s nature as a language-based AI model, which is aimed at providing semantically relevant content based on an underlying associative and stochastic process. Most problems encountered are likely because the model is under-resourced for specific topics (ie, representational bias) and by design attempts to find related content from superficially similar topics rather than returning a null result (ie, learning bias). 10 This leads to presentation of semantically related but clinically irrelevant or incorrect information. Manifestations of this include erroneous conflation of concepts, such as inappropriately combining diagnostic and therapeutic indications for endoscopic procedures (eg, for ERCP) and “hallucination” (presentation of completely nonfactual content as if it were true), such as presenting laparoscopic surgery as a diagnostic alternative to ERCP. Further signs of such an under-resourced topic would be run-to-run variation in content, such as for ERCP where different incorrect indications were presented for each iteration. This would indicate that the model is trying to associate off-topic information to formulate responses and that the process is occurring randomly with each iteration. This phenomenon was present to a lesser degree for EUS, where a small number of incorrect indications were presented. In contrast, EGD and colonoscopy are likely well-resourced topics, where the content is consistently relevant and variations from run-to-run are mainly in the phrasing or language. We further postulate that some errors can be because a topic is over-resourced with irrelevant information. For example, obsolete information (eg, indications for ERCP that have fallen out of favor) was presented alongside current information. This is a form of historical bias 10 that can be potentially harmful as the standards of medical care will shift over time. This could potentially be overcome with appropriate prompt engineering, by specifically directing the model to restrict information provided to more recent times. However, special attention to well-structured prompts may not be a prominent consideration for patients accessing such chatbots. In view of the above limitations with respect to misinformation, it is incumbent on healthcare providers to counsel patients appropriately on misinformation potential. Unfiltered ChatGPT output, without specific prompting or guardrails, can contain highly inaccurate statements, such as in the case of ERCP. In addition to the misinformation issues, providers must also take into account how ChatGPT fits into current clinical practice and what it cannot replace. For example, the provision of information by a chatbot, no matter how advanced, cannot replace the informed consent process. Even if a patient has had information interactively provided by a chatbot, the provider must assess the patient’s understanding and retention of the information provided, which current chatbots are not intentionally designed to assess. Furthermore, providers should be cautious about cybersecurity risks, such as the potential for data interception when communications are made with the external OpenAI server. Although our current example of patient information provision contains no actual patient data, it is conceivable that patients could inadvertently disclose their sensitive health information if they chose to consult a chatbot. We suggest that to further the safe use of AI chatbots to deliver advice on GI endoscopy (or indeed any medical procedure), it is paramount that clinicians give relevant input to data scientists developing the model. In particular, clinicians should assist data scientists with curation of the language corpus for outdated information and ensure that less-represented but relevant topics have sufficient augmentation of training data. For example, although ChatGPT recommended stopping antithrombotics in most iterations, current guidelines allow continuation of aspirin monotherapy periprocedurally for selected patients and GI endoscopic procedures. 11 , 12 For models that are in production, clinicians should evaluate responses for factual accuracy. Finally, whether or not a patient has made use of an AI chatbot for information, clinicians must remain responsible for the patient’s overall understanding of their procedure. Overall, large language models are likely to be of use in increasing patient access to healthcare information. Curation is required to keep healthcare information provided relevant and factually correct. Healthcare providers must be aware of these limitations if using large language models as chatbots in patient care and that they can be potential sources of misinformation if used by patients independently.

The following author disclosed financial relationships: D. M. Y. Tan: Consultant and speaker for Boston Scientific. All other authors disclosed no financial relationships.