METHODS: Two content experts reached consensus on 20 typical patient questions. We retrieved five separate responses for each question from ChatGPT, totaling 100 responses. Each response was evaluated on a 5-point Likert-scale on accuracy by each content expert in an individual fashion.
RESULTS: The two experts were in fair/substantial agreement in the evaluation of responses (Cronbach\'s alpha: 0.64). Of the 100 responses, 17 were relevant and without any inaccuracies, 78 were relevant and with inaccuracies not being harmful, and five were relevant and with inaccuracies potentially harmful. The lowest accuracy scores were obtained for questions dealing with treatment and prognosis.
CONCLUSIONS: ChatGPT often provides relevant answers for patient questions on ODD, but inaccuracies become potentially harmful when questions deal with treatment and prognosis.
方法:两位内容专家就20个典型患者问题达成共识。我们从ChatGPT为每个问题检索了五个单独的回答,共100个回复。每个内容专家以个人方式在5点Likert量表上评估每个响应的准确性。
结果:两位专家在评估回答时达成了相当/实质性的一致(Cronbach的alpha:0.64)。在100个回答中,17个是相关的,没有任何不准确之处,78个是相关的,不准确的是无害的,五个是相关的,并且不准确,可能有害。对于处理治疗和预后的问题,获得了最低的准确性得分。
结论:ChatGPT通常为患者关于ODD的问题提供相关答案,但是当问题涉及治疗和预后时,不准确会变得潜在有害。