{Reference Type}: Journal Article
{Title}: Performance and Consistency of ChatGPT-4 Versus Otolaryngologists: A Clinical Case Series.
{Author}: Lechien JR;Naunheim MR;Maniaci A;Radulesco T;Saibene AM;Chiesa-Estomba CM;Vaira LA;
{Journal}: Otolaryngol Head Neck Surg
{Volume}: 170
{Issue}: 6
{Year}: 2024 Jun 9
{Factor}: 5.591
{DOI}: 10.1002/ohn.759
{Abstract}: <B>OBJECTIVE: </B>To study the performance of Chatbot Generative Pretrained Transformer-4 (ChatGPT-4) in the management of cases in otolaryngology-head and neck surgery.<BR><B>METHODS: </B>Prospective case series.<BR><B>METHODS: </B>Multicenter University Hospitals.<BR><B>METHODS: </B>History, clinical, physical, and additional examinations of adult outpatients consulting in otolaryngology departments of CHU Saint-Pierre and Dour Medical Center were presented to ChatGPT-4, which was interrogated for differential diagnoses, management, and treatment(s). According to specialty, the ChatGPT-4 responses were assessed by 2 distinct, blinded board-certified otolaryngologists with the Artificial Intelligence Performance Instrument.<BR><B>RESULTS: </B>One hundred cases were presented to ChatGPT-4. ChaGPT-4 indicated a mean of 3.34 (95% confidence interval [CI]: 3.09, 3.59) additional examinations per patient versus 2.10 (95% CI: 1.76, 2.34; P = .001) for the practitioners. There was strong consistency (k > 0.600) between otolaryngologists and ChatGPT-4 for the indication of upper aerodigestive tract endoscopy, positron emission tomography and computed tomography, audiometry, tympanometry, and psychophysical evaluations. Primary diagnosis was correctly performed by ChatGPT-4 in 38% to 86% of cases depending on subspecialty. Additional examinations indicated by ChatGPT-4 were pertinent and necessary in 8% to 31% of cases, while the treatment regimen was pertinent in 12% to 44% of cases. The performance of ChatGPT-4 was not influenced by the human-reported level of difficulty of clinical cases.<BR><B>CONCLUSIONS: </B>ChatGPT-4 may be a promising adjunctive tool in otolaryngology, providing extensive documentation about additional examinations, primary and differential diagnoses, and treatments. The ChatGPT-4 is more effective in providing a primary diagnosis, and less effective in the selection of additional examinations and treatments.