PDF(Pubmed)
Abstract：
BACKGROUND: There are approximately 8,000 different rare diseases that affect roughly 400 million people worldwide. Many of them suffer from delayed diagnosis. Ciliopathies are rare monogenic disorders characterized by a significant phenotypic and genetic heterogeneity that raises an important challenge for clinical diagnosis. Diagnosis support systems (DSS) applied to electronic health record (EHR) data may help identify undiagnosed patients, which is of paramount importance to improve patients\' care. Our objective was to evaluate three online-accessible rare disease DSSs using phenotypes derived from EHRs for the diagnosis of ciliopathies.
METHODS: Two datasets of ciliopathy cases, either proven or suspected, and two datasets of controls were used to evaluate the DSSs. Patient phenotypes were automatically extracted from their EHRs and converted to Human Phenotype Ontology terms. We tested the ability of the DSSs to diagnose cases in contrast to controls based on Orphanet ontology.
RESULTS: A total of 79 cases and 38 controls were selected. Performances of the DSSs on ciliopathy real world data (best DSS with area under the ROC curve = 0.72) were not as good as published performances on the test set used in the DSS development phase. None of these systems obtained results which could be described as \"expert-level\". Patients with multisystemic symptoms were generally easier to diagnose than patients with isolated symptoms. Diseases easily confused with ciliopathy generally affected multiple organs and had overlapping phenotypes. Four challenges need to be considered to improve the performances: to make the DSSs interoperable with EHR systems, to validate the performances in real-life settings, to deal with data quality, and to leverage methods and resources for rare and complex diseases.
CONCLUSIONS: Our study provides insights into the complexities of diagnosing highly heterogenous rare diseases and offers lessons derived from evaluation existing DSSs in real-world settings. These insights are not only beneficial for ciliopathy diagnosis but also hold relevance for the enhancement of DSS for various complex rare disorders, by guiding the development of more clinically relevant rare disease DSSs, that could support early diagnosis and finally make more patients eligible for treatment.
METHODS: Two datasets of ciliopathy cases, either proven or suspected, and two datasets of controls were used to evaluate the DSSs. Patient phenotypes were automatically extracted from their EHRs and converted to Human Phenotype Ontology terms. We tested the ability of the DSSs to diagnose cases in contrast to controls based on Orphanet ontology.
RESULTS: A total of 79 cases and 38 controls were selected. Performances of the DSSs on ciliopathy real world data (best DSS with area under the ROC curve = 0.72) were not as good as published performances on the test set used in the DSS development phase. None of these systems obtained results which could be described as \"expert-level\". Patients with multisystemic symptoms were generally easier to diagnose than patients with isolated symptoms. Diseases easily confused with ciliopathy generally affected multiple organs and had overlapping phenotypes. Four challenges need to be considered to improve the performances: to make the DSSs interoperable with EHR systems, to validate the performances in real-life settings, to deal with data quality, and to leverage methods and resources for rare and complex diseases.
CONCLUSIONS: Our study provides insights into the complexities of diagnosing highly heterogenous rare diseases and offers lessons derived from evaluation existing DSSs in real-world settings. These insights are not only beneficial for ciliopathy diagnosis but also hold relevance for the enhancement of DSS for various complex rare disorders, by guiding the development of more clinically relevant rare disease DSSs, that could support early diagnosis and finally make more patients eligible for treatment.
摘要：
背景：大约有8,000种不同的罕见疾病影响全球约4亿人。他们中的许多人患有延迟诊断。纤毛病是罕见的单基因疾病，其特征是具有明显的表型和遗传异质性，这对临床诊断提出了重要挑战。应用于电子健康记录（EHR）数据的诊断支持系统（DSS）可能有助于识别未确诊的患者。这对改善患者护理至关重要。我们的目标是使用来自EHR的表型评估三种在线可访问的罕见疾病DSS，以诊断纤毛病。
方法：两个病例数据集，证明或怀疑，并使用两个对照数据集来评估DSS。患者表型从其EHR中自动提取并转化为人类表型本体论术语。与基于Orphanet本体的控件相比，我们测试了DSS诊断病例的能力。
结果：共选择79例病例和38例对照。DSS对纤毛病现实世界数据的性能（ROC曲线下面积为0.72的最佳DSS）不如DSS开发阶段中使用的测试集上的公开性能。这些系统都没有获得可以描述为“专家级”的结果。具有多系统症状的患者通常比具有孤立症状的患者更容易诊断。容易与纤毛病混淆的疾病通常会影响多个器官，并且表型重叠。需要考虑四个挑战来提高性能：使DSS与EHR系统互操作，为了验证现实生活中的表现，为了处理数据质量，并利用方法和资源来治疗罕见和复杂的疾病。
结论：我们的研究提供了诊断高度异质性罕见疾病的复杂性的见解，并提供了在现实世界中评估现有DSS的经验教训。这些见解不仅有利于纤毛病的诊断，而且与增强DSS治疗各种复杂罕见疾病有关。通过指导更多临床相关的罕见疾病DSS的开发，这可以支持早期诊断，最终使更多的患者有资格接受治疗。
方法：两个病例数据集，证明或怀疑，并使用两个对照数据集来评估DSS。患者表型从其EHR中自动提取并转化为人类表型本体论术语。与基于Orphanet本体的控件相比，我们测试了DSS诊断病例的能力。
结果：共选择79例病例和38例对照。DSS对纤毛病现实世界数据的性能（ROC曲线下面积为0.72的最佳DSS）不如DSS开发阶段中使用的测试集上的公开性能。这些系统都没有获得可以描述为“专家级”的结果。具有多系统症状的患者通常比具有孤立症状的患者更容易诊断。容易与纤毛病混淆的疾病通常会影响多个器官，并且表型重叠。需要考虑四个挑战来提高性能：使DSS与EHR系统互操作，为了验证现实生活中的表现，为了处理数据质量，并利用方法和资源来治疗罕见和复杂的疾病。
结论：我们的研究提供了诊断高度异质性罕见疾病的复杂性的见解，并提供了在现实世界中评估现有DSS的经验教训。这些见解不仅有利于纤毛病的诊断，而且与增强DSS治疗各种复杂罕见疾病有关。通过指导更多临床相关的罕见疾病DSS的开发，这可以支持早期诊断，最终使更多的患者有资格接受治疗。
