目标:自然语言处理和机器学习的最新突破,以ChatGPT为例,刺激了医疗保健的范式转变。由OpenAI于2022年11月发布,ChatGPT迅速获得全球关注。对大量文本数据集进行培训,这个庞大的语言模型拥有巨大的潜力来彻底改变医疗保健。然而,现有文献往往忽视了严格验证和现实适用性的需要。
方法:这项头对头比较研究评估了ChatGPT在提供头颈癌治疗建议方面的能力。模拟每个NCCN指南方案。ChatGPT在初级治疗上被查询,辅助治疗,和后续行动,与NCCN指南相比。性能指标,包括灵敏度,特异性,和F1得分,被用于评估。
结果:该研究包括68例假设病例和204例临床病例。ChatGPT在解决NCCN相关查询方面表现出很有前途的能力,在初级治疗中实现高灵敏度和整体准确性,辅助治疗,和后续行动。该研究的指标在提供相关建议方面表现出了稳健性。然而,注意到一些不准确的地方,尤其是在初级治疗方案中。
结论:我们的研究强调了ChatGPT在提供治疗建议方面的熟练程度。该模型与NCCN指南的一致性为对AI在肿瘤决策支持中不断发展的角色进行细微差别的探索奠定了基础。然而,与AI在临床决策中的可解释性以及临床医生理解AI模型的基本原理的重要性相关的挑战仍未被探索。随着AI的不断进步,模型和医学专家之间的合作努力被认为对于解锁个性化癌症护理的新领域至关重要。
OBJECTIVE: Recent breakthroughs in natural language processing and machine learning, exemplified by ChatGPT, have spurred a paradigm shift in healthcare. Released by OpenAI in November 2022, ChatGPT rapidly gained global attention. Trained on massive text datasets, this large language model holds immense potential to revolutionize healthcare. However, existing literature often overlooks the need for rigorous validation and real-world applicability.
METHODS: This head-to-head comparative study assesses ChatGPT\'s capabilities in providing therapeutic recommendations for head and neck cancers. Simulating every NCCN
Guidelines scenarios. ChatGPT is queried on primary treatments, adjuvant treatment, and follow-up, with responses compared to the NCCN
Guidelines. Performance metrics, including sensitivity, specificity, and F1 score, are employed for assessment.
RESULTS: The study includes 68 hypothetical cases and 204 clinical scenarios. ChatGPT exhibits promising capabilities in addressing NCCN-related queries, achieving high sensitivity and overall accuracy across primary treatment, adjuvant treatment, and follow-up. The study\'s metrics showcase robustness in providing relevant suggestions. However, a few inaccuracies are noted, especially in primary treatment scenarios.
CONCLUSIONS: Our study highlights the proficiency of ChatGPT in providing treatment suggestions. The model\'s alignment with the NCCN
Guidelines sets the stage for a nuanced exploration of AI\'s evolving role in oncological decision support. However, challenges related to the interpretability of AI in clinical decision-making and the importance of clinicians understanding the underlying principles of AI models remain unexplored. As AI continues to advance, collaborative efforts between models and medical experts are deemed essential for unlocking new frontiers in personalized cancer care.