METHODS: We directed inquiries to four distinct AI chatbots to assess their responses in relation to guideline adherence. A total of 115 recommendations were transformed into questions, and responses were evaluated by two urologists with a minimum of 5 years of experience using a 5-point Likert scale (1 - False, 2 - Inadequate, 3 - Sufficient, 4 - Correct, and 5 - Very correct).
RESULTS: The mean scores for Perplexity and ChatGPT 4.0 were 4.68 (SD: 0.80) and 4.80 (SD: 0.47), respectively, both significantly differed the scores of Bing and Bard (Bing vs. Perplexity, P<0.001; Bard vs. Perplexity, P<0.001; Bing vs. ChatGPT, P<0.001; Bard vs. ChatGPT, P<0.001). Bing had a mean score of 4.21 (SD: 0.96), while Bard scored 3.56 (SD: 1.14), with a significant difference (Bing vs. Bard, P<0.001). Bard exhibited the lowest score among all chatbots. Analysis of references revealed that Perplexity and Bing cited the guideline most frequently (47.3% and 30%, respectively).
CONCLUSIONS: Our findings demonstrate that ChatGPT 4.0 and, notably, Perplexity align well with EAU guideline recommendations. These continuously evolving applications may play a crucial role in delivering information to physicians in the future, especially for urolithiasis.
方法:我们针对四个不同的AI聊天机器人进行了调查,以评估它们对指南依从性的反应。共有115项建议转化为问题,和反应由两名至少5年经验的泌尿科医师使用5分Likert量表(1-False,2-不足,3-足够,4-正确,和5-非常正确)。
结果:困惑和ChatGPT4.0的平均得分为4.68(SD:0.80)和4.80(SD:0.47),分别,两者的得分都有显著不同的Bing和Bard(Bing与困惑,p<.001;吟游诗人与困惑,p<.001;Bingvs.ChatGPT,p<.001;吟游诗人与ChatGPT,p<.001)。Bing的平均得分为4.21(SD:0.96),而巴德得了3.56分(标准差:1.14),具有显著差异(Bing与巴德,p<.001)。巴德在所有聊天机器人中得分最低。参考文献分析表明,困惑和必应引用指南的频率最高(47.3%和30%,分别)。
结论:我们的研究结果表明,ChatGPT4.0和,特别是,困惑与EAU指南建议一致。这些不断发展的应用程序可能在将来向医生提供信息方面发挥关键作用。尤其是尿石症.