■为了检查使用基于临床的大语言模型(LLM)的自然语言处理(NLP)是否可用于从常规可用的自由文本放射学报告中预测全髋关节或全膝关节置换术(THA/TKA)的患者选择。
■根据人工智能进行了数据预处理和分析,以彻底改变髋关节和膝关节(ARCHERY)项目方案中的患者护理途径。ThisincludinguseofdeidentifiedScotishregionalclinicaldataofpatientsreferredforconsiderationofTHA/TKA,保存在为人工智能(AI)推理设计的安全数据环境中。仅包括术前放射学报告。NLP算法基于免费提供的GatorTron模型,LLM接受了超过820亿字的去识别临床文本的培训。执行了两个推理任务:模型微调后的评估(50个周期和三个周期的k折交叉验证),和外部验证。
■对于THA,包括5558例患者放射学报告,其中4137个用于模型训练和测试,和1,421用于外部验证。培训后,模型性能证明了平均(三次折叠的平均值)精度,F1得分,和受试者工作曲线下面积(AUROC)值为0.850(95%置信区间(CI)0.833至0.867),0.813(95%CI0.785至0.841),和0.847(95%CI0.822至0.872),分别。对于TKA,包括7,457例患者放射学报告,有3478个用于模型训练和测试,和3,152用于外部验证。性能指标包括准确性、F1得分,AUROC值为0.757(95%CI为0.702至0.811),0.543(95%CI0.479至0.607),和0.717(95%CI0.657至0.778)。在两个队列中,外部验证的性能均显着下降。
■使用常规可用的术前放射学报告提供了有希望的潜力,可以帮助筛选THA的合适候选者。但不是为了TKA.外部验证结果表明,当面对新的临床队列时,进一步进行模型测试和培训的重要性。
UNASSIGNED: To examine whether natural language processing (NLP) using a clinically based large language model (LLM) could be used to predict patient selection for total hip or total knee arthroplasty (THA/TKA) from routinely available free-text radiology
reports.
UNASSIGNED: Data pre-processing and analyses were conducted according to the Artificial intelligence to Revolutionize the patient Care pathway in Hip and knEe aRthroplastY (ARCHERY) project protocol. This included use of de-identified Scottish regional clinical data of patients referred for consideration of THA/TKA, held in a secure data environment designed for artificial intelligence (AI) inference. Only preoperative radiology
reports were included. NLP algorithms were based on the freely available GatorTron model, a LLM trained on over 82 billion words of de-identified clinical text. Two inference tasks were performed: assessment after model-fine tuning (50 Epochs and three cycles of k-fold cross validation), and external validation.
UNASSIGNED: For THA, there were 5,558 patient radiology
reports included, of which 4,137 were used for model training and testing, and 1,421 for external validation. Following training, model performance demonstrated average (mean across three folds) accuracy, F1 score, and area under the receiver operating curve (AUROC) values of 0.850 (95% confidence interval (CI) 0.833 to 0.867), 0.813 (95% CI 0.785 to 0.841), and 0.847 (95% CI 0.822 to 0.872), respectively. For TKA, 7,457 patient radiology
reports were included, with 3,478 used for model training and testing, and 3,152 for external validation. Performance metrics included accuracy, F1 score, and AUROC values of 0.757 (95% CI 0.702 to 0.811), 0.543 (95% CI 0.479 to 0.607), and 0.717 (95% CI 0.657 to 0.778) respectively. There was a notable deterioration in performance on external validation in both cohorts.
UNASSIGNED: The use of routinely available preoperative radiology
reports provides promising potential to help screen suitable candidates for THA, but not for TKA. The external validation results demonstrate the importance of further model testing and training when confronted with new clinical cohorts.