Artificial intelligence in practice: measuring its medical accuracy in oculoplastics consultations

Adam J. Neuhouser, Alisha Kamboj, Ali Mokhtarzadeh, Andrew R Harrison

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: The aim of this study was to investigate the medical accuracy of responses produced by Chat Generative Pretrained Transformer 4 (Chat GPT-4) and DALLE-2 in relation to common questions encountered during oculoplastic consultations. Methods: The 5 most frequently discussed oculoplastic procedures on social media were selected for evaluation using Chat GPT-4 and DALLE-2. Questions were formulated from common patient concerns and inputted into Chat GPT-4, and responses were assessed on a 3-point scale. For procedure imagery, descriptions were submitted to DALLE-2, and the resulted images were graded for anatomical and surgical accuracy. Grading was completed by 5 oculoplastic surgeons through a 110-question survey. Results: Overall, 87.3% of Chat GPT-4’s responses achieved a score of 2 or 3 points, denoting a good to high level of accuracy. Across all procedures, questions about pain, bruising, procedure risk, and adverse events garnered high scores. Conversely, responses regarding specific case scenarios, procedure longevity, and procedure definitions were less accurate. Images produced by DALLE-2-were notably subpar, often failing to accurately depict surgical outcomes and realistic details. Conclusions: Chat GPT-4 demonstrated a creditable level of accuracy in addressing common oculoplastic procedure concerns. However, its limitations in handling case-based scenarios suggests that it is best suited as a supplementary source of information rather than a primary diagnostic or consultative tool. The current state of medical imagery generated by means of artificial intelligence lacks anatomical accuracy. Significant technological advancements are necessary before such imagery can complement oculoplastic consultations effectively.

Original languageEnglish (US)
JournalJournal for Modeling in Ophthalmology
Volume6
Issue number1
DOIs
StatePublished - Feb 13 2024

Bibliographical note

Publisher Copyright:
© Modeling and Artificial Intelligence in Ophthalmology 2024.

Keywords

  • artificial intelligence
  • Chat GPT
  • DALLE
  • oculoplastics
  • patient information

Fingerprint

Dive into the research topics of 'Artificial intelligence in practice: measuring its medical accuracy in oculoplastics consultations'. Together they form a unique fingerprint.

Cite this