Original Research
Tuesday, October 15, 2024
08:30 AM–10:00 AM
Abstract
Introduction: ChatGPT has been used widely and shown promise in healthcare settings. However, the performance of ChatGPT for the treatment of infectious diseases in the vulnerable populations needs investigation.
Research Question or Hypothesis: How does ChatGPT perform for the treatment of infectious diseases in vulnerable populations?
Study Design: Cross-sectional observational study
Methods: We designed 37 mock cases on how to treat common infectious diseases such as urinary tract infection, pneumonia, and skin infection in vulnerable populations including pregnant women, the elderly, children, patients with kidney or liver diseases, and those with a history of antibiotic allergy. Each query was posed to ChatGPT-4 six times to determine the selection of antibiotics, dosage, and treatment duration by two researchers independently. The answers were then evaluated separately by them. The evaluation system for responses was as follows: 1. correct and comprehensive; 2. comprehensive and partially correct; 3. correct and not comprehensive; 4. partially correct and not comprehensive; 5. mixed with correct and incorrect answers. Percentages were used to describe the results.
Results: 38% of responses were comprehensive and correct, with 12% containing errors in pregnant women. 36% of responses were comprehensive and correct, and 18% contained errors for allergy-related questions. For the elderly, about 28% of responses were comprehensive and correct, while 25% contained errors. However, the error rate was highest for responses related to patients with kidney disease, at 79%, with almost no comprehensive and correct answers, particularly with incorrect dose and administration interval adjustments based on creatinine clearance. For children, nearly 18% of responses were comprehensive and correct, while 44% contained errors.
Conclusion: Our study showed that the current version of ChatGPT-4 would not be appropriate for the treatment of infectious diseases in vulnerable populations. Thus, recommendations generated by ChatGPT would need to be evaluated by healthcare professionals to ensure accuracy and appropriateness prior to use in patient care.
Presenting Author
Heqing Tao MDDepartment of Gastroenterology, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou Medical University
Authors
Jiayu Cui PharmD
CVS Health
Ligang Liu PharmD
Use login from my institution
Milap C. Nahata PharmD, MS
The Ohio State University College of Pharmacy