PDF(Pubmed)
Abstract：
BACKGROUND: Spondyloarthritis (SpA), a chronic inflammatory disorder, predominantly impacts the sacroiliac joints and spine, significantly escalating the risk of disability. SpA\'s complexity, as evidenced by its diverse clinical presentations and symptoms that often mimic other diseases, presents substantial challenges in its accurate diagnosis and differentiation. This complexity becomes even more pronounced in nonspecialist health care environments due to limited resources, resulting in delayed referrals, increased misdiagnosis rates, and exacerbated disability outcomes for patients with SpA. The emergence of large language models (LLMs) in medical diagnostics introduces a revolutionary potential to overcome these diagnostic hurdles. Despite recent advancements in artificial intelligence and LLMs demonstrating effectiveness in diagnosing and treating various diseases, their application in SpA remains underdeveloped. Currently, there is a notable absence of SpA-specific LLMs and an established benchmark for assessing the performance of such models in this particular field.
OBJECTIVE: Our objective is to develop a foundational medical model, creating a comprehensive evaluation benchmark tailored to the essential medical knowledge of SpA and its unique diagnostic and treatment protocols. The model, post-pretraining, will be subject to further enhancement through supervised fine-tuning. It is projected to significantly aid physicians in SpA diagnosis and treatment, especially in settings with limited access to specialized care. Furthermore, this initiative is poised to promote early and accurate SpA detection at the primary care level, thereby diminishing the risks associated with delayed or incorrect diagnoses.
METHODS: A rigorous benchmark, comprising 222 meticulously formulated multiple-choice questions on SpA, will be established and developed. These questions will be extensively revised to ensure their suitability for accurately evaluating LLMs\' performance in real-world diagnostic and therapeutic scenarios. Our methodology involves selecting and refining top foundational models using public data sets. The best-performing model in our benchmark will undergo further training. Subsequently, more than 80,000 real-world inpatient and outpatient cases from hospitals will enhance LLM training, incorporating techniques such as supervised fine-tuning and low-rank adaptation. We will rigorously assess the models\' generated responses for accuracy and evaluate their reasoning processes using the metrics of fluency, relevance, completeness, and medical proficiency.
RESULTS: Development of the model is progressing, with significant enhancements anticipated by early 2024. The benchmark, along with the results of evaluations, is expected to be released in the second quarter of 2024.
CONCLUSIONS: Our trained model aims to capitalize on the capabilities of LLMs in analyzing complex clinical data, thereby enabling precise detection, diagnosis, and treatment of SpA. This innovation is anticipated to play a vital role in diminishing the disabilities arising from delayed or incorrect SpA diagnoses. By promoting this model across diverse health care settings, we anticipate a significant improvement in SpA management, culminating in enhanced patient outcomes and a reduced overall burden of the disease.
UNASSIGNED: DERR1-10.2196/57001.
OBJECTIVE: Our objective is to develop a foundational medical model, creating a comprehensive evaluation benchmark tailored to the essential medical knowledge of SpA and its unique diagnostic and treatment protocols. The model, post-pretraining, will be subject to further enhancement through supervised fine-tuning. It is projected to significantly aid physicians in SpA diagnosis and treatment, especially in settings with limited access to specialized care. Furthermore, this initiative is poised to promote early and accurate SpA detection at the primary care level, thereby diminishing the risks associated with delayed or incorrect diagnoses.
METHODS: A rigorous benchmark, comprising 222 meticulously formulated multiple-choice questions on SpA, will be established and developed. These questions will be extensively revised to ensure their suitability for accurately evaluating LLMs\' performance in real-world diagnostic and therapeutic scenarios. Our methodology involves selecting and refining top foundational models using public data sets. The best-performing model in our benchmark will undergo further training. Subsequently, more than 80,000 real-world inpatient and outpatient cases from hospitals will enhance LLM training, incorporating techniques such as supervised fine-tuning and low-rank adaptation. We will rigorously assess the models\' generated responses for accuracy and evaluate their reasoning processes using the metrics of fluency, relevance, completeness, and medical proficiency.
RESULTS: Development of the model is progressing, with significant enhancements anticipated by early 2024. The benchmark, along with the results of evaluations, is expected to be released in the second quarter of 2024.
CONCLUSIONS: Our trained model aims to capitalize on the capabilities of LLMs in analyzing complex clinical data, thereby enabling precise detection, diagnosis, and treatment of SpA. This innovation is anticipated to play a vital role in diminishing the disabilities arising from delayed or incorrect SpA diagnoses. By promoting this model across diverse health care settings, we anticipate a significant improvement in SpA management, culminating in enhanced patient outcomes and a reduced overall burden of the disease.
UNASSIGNED: DERR1-10.2196/57001.
摘要：
背景：脊柱关节炎（SpA），慢性炎症性疾病,主要影响骶髂关节和脊柱，显着增加残疾的风险。SpA的复杂性，其多样化的临床表现和症状往往模仿其他疾病，对其准确诊断和鉴别提出了重大挑战。由于资源有限，这种复杂性在非专业医疗保健环境中变得更加明显，导致延迟的推荐，误诊率增加，并加剧了SpA患者的残疾结果。医学诊断中大型语言模型（LLM）的出现为克服这些诊断障碍带来了革命性的潜力。尽管人工智能和LLM的最新进展证明了诊断和治疗各种疾病的有效性，它们在SpA中的应用仍然不发达。目前,有一个明显的缺乏SpA特定的LLM和一个既定的基准来评估这种模型在这个特定领域的性能。
目的：我们的目标是建立基础医学模式，创建针对SpA的基本医学知识及其独特的诊断和治疗方案的全面评估基准。模型,培训后，将通过监督微调进一步增强。预计将大大帮助医生进行SpA诊断和治疗，特别是在获得专门护理的机会有限的环境中。此外，这一举措有望在初级保健层面促进早期和准确的SpA检测，从而减少与延迟或错误诊断相关的风险。
方法：严格的基准，包括222个精心制定的SpA多项选择题，将被建立和发展。这些问题将被广泛修订，以确保它们在现实世界的诊断和治疗方案中准确评估LLM的表现。我们的方法涉及使用公共数据集选择和完善顶级基础模型。我们基准测试中表现最好的模型将接受进一步的培训。随后，来自医院的80,000多例实际住院和门诊病例将加强LLM培训，结合监督微调和低秩适应等技术。我们将严格评估模型生成的响应的准确性，并使用流畅性指标评估其推理过程，相关性，完整性，和医疗水平。
结果：模型的开发正在进行中，预计到2024年初将有显著的增强。基准，以及评估结果，预计将于2024年第二季度发布。
结论：我们的训练模型旨在利用LLM分析复杂临床数据的能力，从而实现精确检测，诊断,和治疗SpA。预计这项创新将在减少因延迟或错误的SpA诊断引起的残疾方面发挥至关重要的作用。通过在不同的医疗保健环境中推广这种模式，我们预计SpA管理会有显著改善，最终提高了患者的预后，减轻了疾病的总体负担。
■DERR1-10.2196/57001。
目的：我们的目标是建立基础医学模式，创建针对SpA的基本医学知识及其独特的诊断和治疗方案的全面评估基准。模型,培训后，将通过监督微调进一步增强。预计将大大帮助医生进行SpA诊断和治疗，特别是在获得专门护理的机会有限的环境中。此外，这一举措有望在初级保健层面促进早期和准确的SpA检测，从而减少与延迟或错误诊断相关的风险。
方法：严格的基准，包括222个精心制定的SpA多项选择题，将被建立和发展。这些问题将被广泛修订，以确保它们在现实世界的诊断和治疗方案中准确评估LLM的表现。我们的方法涉及使用公共数据集选择和完善顶级基础模型。我们基准测试中表现最好的模型将接受进一步的培训。随后，来自医院的80,000多例实际住院和门诊病例将加强LLM培训，结合监督微调和低秩适应等技术。我们将严格评估模型生成的响应的准确性，并使用流畅性指标评估其推理过程，相关性，完整性，和医疗水平。
结果：模型的开发正在进行中，预计到2024年初将有显著的增强。基准，以及评估结果，预计将于2024年第二季度发布。
结论：我们的训练模型旨在利用LLM分析复杂临床数据的能力，从而实现精确检测，诊断,和治疗SpA。预计这项创新将在减少因延迟或错误的SpA诊断引起的残疾方面发挥至关重要的作用。通过在不同的医疗保健环境中推广这种模式，我们预计SpA管理会有显著改善，最终提高了患者的预后，减轻了疾病的总体负担。
■DERR1-10.2196/57001。
