背景:随着人工智能(AI)的兴起,目前尚不清楚AI是否能够专业评估医学研究并给出科学有效的建议。
目的:本研究旨在评估ChatGPT对脑脓肿诊断和治疗的十个关键问题的回答与欧洲临床微生物学和传染病学会(ESCMID)最近发布的指南相比的准确性。
方法:所有十个PECO(人口,曝光,比较器,结果)在指南过程中提出的问题直接提交给ChatGPT。接下来,ChatGPT还接受了ESCMID委员会为每个PECO问题选择的研究数据。随后将AI的回答与ESCMID指南的建议进行了比较。
结果:对于20项挑战中的17项,ChatGPT能够提供有关脑脓肿患者管理的建议,包括证据等级和推荐强度。没有数据提示,70%的问题回答与指南建议非常相似。在与指南建议不同的答案中,不存在患者危险.数据输入略微提高了ChatGPT建议的清晰度,但是,然而,导致不太正确的答案,包括两个直接与准则矛盾的建议,与对患者有危险的可能性有关。
结论:ChatGPT似乎能够快速收集有关脑脓肿的信息,并就大多数情况下有关其管理的关键问题提出建议。然而,单一的反应可能会伤害患者。因此,专家委员会的专业知识仍然不可避免。
BACKGROUND: With artificial intelligence (AI) on the rise, it remains unclear if AI is able to professionally evaluate medical research and give scientifically valid recommendations.
OBJECTIVE: This study aimed to assess the accuracy of ChatGPT\'s responses to ten key questions on brain abscess diagnostics and treatment in comparison to the
guideline recently published by the European Society for Clinical Microbiology and Infectious Diseases (ESCMID).
METHODS: All ten PECO (Population, Exposure, Comparator, Outcome) questions which had been developed during the
guideline process were presented directly to ChatGPT. Next, ChatGPT was additionally fed with data from studies selected for each PECO question by the ESCMID committee. AI\'s responses were subsequently compared with the recommendations of the ESCMID
guideline.
RESULTS: For 17 out of 20 challenges, ChatGPT was able to give recommendations on the management of patients with brain abscess, including grade of evidence and strength of recommendation. Without data prompting, 70% of questions were answered very similar to the
guideline recommendation. In the answers that differed from the
guideline recommendations, no patient hazard was present. Data input slightly improved the clarity of ChatGPT\'s recommendations, but, however, led to less correct answers including two recommendations that directly contradicted the guideline, being associated with the possibility of a hazard to the patient.
CONCLUSIONS: ChatGPT seems to be able to rapidly gather information on brain abscesses and give recommendations on key questions about their management in most cases. Nevertheless, single responses could possibly harm the patients. Thus, the expertise of an expert committee remains inevitable.