Abstract：
BACKGROUND: Diagnostic errors pose significant health risks and contribute to patient mortality. With the growing accessibility of electronic health records, machine learning models offer a promising avenue for enhancing diagnosis quality. Current research has primarily focused on a limited set of diseases with ample training data, neglecting diagnostic scenarios with limited data availability.
OBJECTIVE: This study aims to develop an information retrieval (IR)-based framework that accommodates data sparsity to facilitate broader diagnostic decision support.
METHODS: We introduced an IR-based diagnostic decision support framework called CliniqIR. It uses clinical text records, the Unified Medical Language System Metathesaurus, and 33 million PubMed abstracts to classify a broad spectrum of diagnoses independent of training data availability. CliniqIR is designed to be compatible with any IR framework. Therefore, we implemented it using both dense and sparse retrieval approaches. We compared CliniqIR\'s performance to that of pretrained clinical transformer models such as Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) in supervised and zero-shot settings. Subsequently, we combined the strength of supervised fine-tuned ClinicalBERT and CliniqIR to build an ensemble framework that delivers state-of-the-art diagnostic predictions.
RESULTS: On a complex diagnosis data set (DC3) without any training data, CliniqIR models returned the correct diagnosis within their top 3 predictions. On the Medical Information Mart for Intensive Care III data set, CliniqIR models surpassed ClinicalBERT in predicting diagnoses with <5 training samples by an average difference in mean reciprocal rank of 0.10. In a zero-shot setting where models received no disease-specific training, CliniqIR still outperformed the pretrained transformer models with a greater mean reciprocal rank of at least 0.10. Furthermore, in most conditions, our ensemble framework surpassed the performance of its individual components, demonstrating its enhanced ability to make precise diagnostic predictions.
CONCLUSIONS: Our experiments highlight the importance of IR in leveraging unstructured knowledge resources to identify infrequently encountered diagnoses. In addition, our ensemble framework benefits from combining the complementary strengths of the supervised and retrieval-based models to diagnose a broad spectrum of diseases.
OBJECTIVE: This study aims to develop an information retrieval (IR)-based framework that accommodates data sparsity to facilitate broader diagnostic decision support.
METHODS: We introduced an IR-based diagnostic decision support framework called CliniqIR. It uses clinical text records, the Unified Medical Language System Metathesaurus, and 33 million PubMed abstracts to classify a broad spectrum of diagnoses independent of training data availability. CliniqIR is designed to be compatible with any IR framework. Therefore, we implemented it using both dense and sparse retrieval approaches. We compared CliniqIR\'s performance to that of pretrained clinical transformer models such as Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) in supervised and zero-shot settings. Subsequently, we combined the strength of supervised fine-tuned ClinicalBERT and CliniqIR to build an ensemble framework that delivers state-of-the-art diagnostic predictions.
RESULTS: On a complex diagnosis data set (DC3) without any training data, CliniqIR models returned the correct diagnosis within their top 3 predictions. On the Medical Information Mart for Intensive Care III data set, CliniqIR models surpassed ClinicalBERT in predicting diagnoses with <5 training samples by an average difference in mean reciprocal rank of 0.10. In a zero-shot setting where models received no disease-specific training, CliniqIR still outperformed the pretrained transformer models with a greater mean reciprocal rank of at least 0.10. Furthermore, in most conditions, our ensemble framework surpassed the performance of its individual components, demonstrating its enhanced ability to make precise diagnostic predictions.
CONCLUSIONS: Our experiments highlight the importance of IR in leveraging unstructured knowledge resources to identify infrequently encountered diagnoses. In addition, our ensemble framework benefits from combining the complementary strengths of the supervised and retrieval-based models to diagnose a broad spectrum of diseases.
摘要：
背景：诊断错误会带来重大的健康风险，并导致患者死亡。随着电子健康记录的日益普及，机器学习模型为提高诊断质量提供了一条有前途的途径。目前的研究主要集中在一组有限的疾病和充足的训练数据，忽略数据可用性有限的诊断方案。
目的：本研究旨在开发一种基于信息检索（IR）的框架，该框架可容纳数据稀疏性，以促进更广泛的诊断决策支持。
方法：我们介绍了一个基于IR的诊断决策支持框架，称为CliniqIR。它使用临床文本记录，统一的医学语言系统词库，和3300万份PubMed摘要，以独立于训练数据可用性对广泛的诊断进行分类。CliniqIR旨在与任何IR框架兼容。因此,我们使用密集和稀疏检索方法实现了它。我们将CliniqIR的性能与预训练的临床变压器模型的性能进行了比较，例如在监督和零射设置下来自变压器的临床双向编码器表示（ClinicalBERT）。随后，我们结合了监督微调ClinicalBERT和CliniqIR的优势,构建了一个集成框架,提供最先进的诊断预测.
结果：在没有任何训练数据的复杂诊断数据集（DC3）上，CliniqIR模型在其前3个预测中返回了正确的诊断。关于重症监护医学信息集市III数据集，CliniqIR模型在预测<5个训练样本的诊断方面超过ClinicalBERT，平均倒数排名差异为0.10。在零射击环境中，模型没有接受疾病特异性训练，CliniqIR仍然优于预训练的变压器模型，其平均倒数排名至少为0.10。此外，在大多数情况下,我们的集成框架超越了其各个组件的性能，证明其增强了做出精确诊断预测的能力。
结论：我们的实验强调了IR在利用非结构化知识资源识别不常遇到的诊断方面的重要性。此外,我们的集成框架受益于结合监督和基于检索的模型的互补优势来诊断广泛的疾病.
目的：本研究旨在开发一种基于信息检索（IR）的框架，该框架可容纳数据稀疏性，以促进更广泛的诊断决策支持。
方法：我们介绍了一个基于IR的诊断决策支持框架，称为CliniqIR。它使用临床文本记录，统一的医学语言系统词库，和3300万份PubMed摘要，以独立于训练数据可用性对广泛的诊断进行分类。CliniqIR旨在与任何IR框架兼容。因此,我们使用密集和稀疏检索方法实现了它。我们将CliniqIR的性能与预训练的临床变压器模型的性能进行了比较，例如在监督和零射设置下来自变压器的临床双向编码器表示（ClinicalBERT）。随后，我们结合了监督微调ClinicalBERT和CliniqIR的优势,构建了一个集成框架,提供最先进的诊断预测.
结果：在没有任何训练数据的复杂诊断数据集（DC3）上，CliniqIR模型在其前3个预测中返回了正确的诊断。关于重症监护医学信息集市III数据集，CliniqIR模型在预测<5个训练样本的诊断方面超过ClinicalBERT，平均倒数排名差异为0.10。在零射击环境中，模型没有接受疾病特异性训练，CliniqIR仍然优于预训练的变压器模型，其平均倒数排名至少为0.10。此外，在大多数情况下,我们的集成框架超越了其各个组件的性能，证明其增强了做出精确诊断预测的能力。
结论：我们的实验强调了IR在利用非结构化知识资源识别不常遇到的诊断方面的重要性。此外,我们的集成框架受益于结合监督和基于检索的模型的互补优势来诊断广泛的疾病.
