Scientific Poster Session IV - Original Research

Abstract

Research Question or Hypothesis: How accurate and reliable are AI tools for solving pharmaceutical calculations?

Study Design: A descriptive study analyzing the accuracy and teachability of AI tools for solving pharmaceutical calculations commonly encountered in the pharmacy curriculum.

Methods: Eleven free-access AI tools with the potential to perform mathematical calculations were gathered through an internet search with the assistance of a librarian. Seven faculty-generated questions were tested with each AI tool: one control, two creatinine clearance, one oral to intravenous dose conversion, one vancomycin pharmacokinetic dosing, one gentamicin dose, and one number needed to harm (NNH). The primary outcome was the AI tools’ ability to perform the calculation correctly by reporting a correct response. Secondary outcomes included types of mistakes made and teachability for each tool.

Results: The control question was answered correctly by 10 (90.9%) AI tools, and all AI tools correctly answered the dose conversion problem. Eight (72.7%) tools were able to calculate NNH. Only one (9.1%) calculated the correct gentamicin dose and interval. None of the tools correctly calculated creatinine clearance or vancomycin dose and interval. The most frequently encountered mistakes were incorrect weight selection for creatinine clearance and use of incorrect formulas. Nine (81.8%) of the tools were able to be taught on at least one question within a session.

Conclusion: AI tools proved to be unreliable for solving complex pharmaceutical calculations that require multiple steps and critical thinking. Certain tools may be more reliable for straightforward calculations such as proportional reasoning or NNH.

Presenting Author

Authors