目标:最近大型语言模型(LLM)的普及激增,比如ChatGPT,展示了他们在体检方面的熟练程度和在医疗保健中的潜在应用。然而,LLM具有固有的局限性,包括不一致的准确性,具体提示要求,以及产生有害幻觉的风险。特定于域的模型可以有效地解决这些限制。
方法:开发设计。
方法:虚拟。
方法:耳鼻咽喉头颈外科(OHNS)相关数据从开放的Internet资源中系统地收集,并编入知识数据库。我们利用检索增强语言建模来回忆这些信息,并将其用于预训练,然后将其集成到ChatGPT4.0中,创建了一个称为ChatENT的OHNS特定知识问答平台。该模型在不同类型的问题上进一步测试。
结果:ChatENT在分析和解释OHNS信息方面表现增强,ChatGPT4.0在加拿大皇家学院OHNS样本考试试题挑战和美国董事会实践试题挑战中的表现均优于ChatGPT4.0,误差减少了58.4%和26.0%,分别。Chatent产生的幻觉较少,并且表现出更大的一致性。
结论:据我们所知,ChatENT是医疗领域中第一个利用最新LLM的专业特定知识检索人工智能。它似乎在医学教育等领域有相当大的前景,患者教育,和临床决策支持。该模型证明了克服现有LLM限制的能力,从而预示着更精确的未来,安全,以及OHNS和其他医疗领域的用户友好应用。
OBJECTIVE: The recent surge in popularity of large language models (LLMs), such as ChatGPT, has showcased their proficiency in medical examinations and potential applications in health care. However, LLMs possess inherent limitations, including inconsistent accuracy, specific prompting requirements, and the risk of generating harmful hallucinations. A domain-specific model might address these limitations effectively.
METHODS: Developmental design.
METHODS: Virtual.
METHODS: Otolaryngology-head and neck surgery (OHNS) relevant data were systematically gathered from open-access Internet sources and indexed into a knowledge database. We leveraged Retrieval-Augmented Language Modeling to recall this information and utilized it for pretraining, which was then integrated into ChatGPT4.0, creating an OHNS-specific knowledge question & answer platform known as ChatENT. The model is further tested on different types of questions.
RESULTS: ChatENT showed enhanced performance in the analysis and interpretation of OHNS information, outperforming ChatGPT4.0 in both the Canadian Royal College OHNS sample examination questions challenge and the US board practice questions challenge, with a 58.4% and 26.0% error reduction, respectively. ChatENT generated fewer hallucinations and demonstrated greater consistency.
CONCLUSIONS: To the best of our knowledge, ChatENT is the first specialty-specific knowledge retrieval artificial intelligence in the medical field that utilizes the latest LLM. It appears to have considerable promise in areas such as medical education, patient education, and clinical decision support. The model has demonstrated the capacity to overcome the limitations of existing LLMs, thereby signaling a future of more precise, safe, and user-friendly applications in the realm of OHNS and other medical fields.