背景:下腰痛(LBP)是一种普遍的医疗保健问题,通常对保守治疗有反应。然而,它也可能源于严重的条件,标有“危险信号”(RF),如恶性肿瘤,马尾综合征,骨折,感染,脊椎关节病,动脉瘤破裂,医生应该对此保持警惕。鉴于对在线健康信息的日益依赖,这项研究评估了ChatGPT-3.5(OpenAI,旧金山,CA,美国)和GoogleBard(谷歌,山景,CA,USA)对RF相关LBP问题的准确性及其辨别病情严重程度的能力。
方法:我们根据LBP指南创建了关于RF相关症状和疾病的70个问题。其中,58例出现单一症状(SS),12例出现LBP的多种症状(MS)。向ChatGPT和GoogleBard提出了问题,两位作者对回答的准确性进行了评估,完整性,和相关性(ACR)使用5点标题标准。
结果:Cohen\的kappa值(0.60-0.81)表明作者之间有很大的一致性。对于58个SS问题,ChatGPT-3.5的平均回答分数为3.47至3.85,GoogleBard的平均回答分数为3.36至3.76,对于12个MS问题,ChatGPT-3.5从4.04到4.29,GoogleBard从3.50到3.71。这些响应的评级范围从“良好”到“优秀”。大多数SS响应有效地传达了情况的严重性(ChatGPT-3.5为93.1%,GoogleBard为94.8%),和所有MS的反应都这样做。ChatGPT-3.5和GoogleBard评分差异无统计学意义(p>0.05)。
结论:在一个以广泛寻求在线健康信息为特征的时代,人工智能(AI)系统在提供精确的医疗信息方面发挥着至关重要的作用。如果这些技术继续改进,它们可能在健康信息领域充满希望。
BACKGROUND: Low back pain (LBP) is a prevalent healthcare concern that is frequently responsive to conservative treatment. However, it can also stem from severe conditions, marked by \'red flags\' (RF) such as malignancy, cauda equina syndrome, fractures, infections, spondyloarthropathies, and aneurysm rupture, which physicians should be vigilant about. Given the increasing reliance on online health information, this study assessed ChatGPT-3.5\'s (OpenAI, San Francisco, CA, USA) and GoogleBard\'s (Google, Mountain View, CA, USA) accuracy in responding to RF-related LBP questions and their capacity to discriminate the severity of the condition.
METHODS: We created 70 questions on RF-related symptoms and diseases following the LBP guidelines. Among them, 58 had a single symptom (SS), and 12 had multiple symptoms (MS) of LBP. Questions were posed to ChatGPT and GoogleBard, and responses were assessed by two authors for accuracy, completeness, and relevance (ACR) using a 5-point rubric criteria.
RESULTS: Cohen\'s kappa values (0.60-0.81) indicated significant agreement among the authors. The average scores for responses ranged from 3.47 to 3.85 for ChatGPT-3.5 and from 3.36 to 3.76 for GoogleBard for 58 SS questions, and from 4.04 to 4.29 for ChatGPT-3.5 and from 3.50 to 3.71 for GoogleBard for 12 MS questions. The ratings for these responses ranged from \'good\' to \'excellent\'. Most SS responses effectively conveyed the severity of the situation (93.1% for ChatGPT-3.5, 94.8% for GoogleBard), and all MS responses did so. No statistically significant differences were found between ChatGPT-3.5 and GoogleBard scores (p>0.05).
CONCLUSIONS: In an era characterized by widespread online health information seeking, artificial intelligence (AI) systems play a vital role in delivering precise medical information. These technologies may hold promise in the field of health information if they continue to improve.