背景:近年来,利用常规收集的医疗保健数据(RCD)的观察性研究的使用趋势越来越明显.这些研究依赖于算法来识别用于统计分析的特定健康状况(例如糖尿病或败血症)。然而,算法的开发和验证有很大的差异,导致性能经常欠佳,并对研究结果的有效性构成重大威胁。不幸的是,这些问题经常被忽视。
方法:我们系统地制定了开发指南,验证,和评估旨在识别健康状况的算法(DEVELOP-RCD)。我们最初的努力包括对与算法开发相关的概念和方法论问题的已发表研究进行叙述性审查和系统审查。验证,和评价。随后,我们对脓毒症的识别算法进行了实证研究.基于这些发现,我们为算法开发制定了具体的工作流程和建议,验证,和指导内的评估。最后,该指南经过了一个由20名外部专家组成的小组的独立审查,然后召开了一次共识会议以最终确定该指南。
结果:算法开发的标准化工作流程,验证,并建立了评价。在特定健康状况考虑的指导下,该工作流程包括四个综合步骤:评估现有算法对目标健康状态的适用性;使用推荐方法开发新算法;使用规定的性能度量验证算法;评估算法对研究结果的影响。此外,提出了13项良好做法建议,并附有详细解释。此外,本指南的应用纳入了一项关于脓毒症鉴别的实际研究.
结论:指南的建立旨在帮助研究人员和临床医生适当和准确地开发和应用从RCD中识别健康状况的算法。本指南有可能提高涉及刚果民盟的观察性研究结果的可信度。
BACKGROUND: In recent years, there has been a growing trend in the utilization of observational studies that make use of routinely collected healthcare data (RCD). These studies rely on algorithms to identify specific health conditions (e.g. diabetes or sepsis) for statistical analyses. However, there has been substantial variation in the algorithm development and validation, leading to frequently suboptimal performance and posing a significant threat to the validity of study findings. Unfortunately, these issues are often overlooked.
METHODS: We systematically developed
guidance for the development, validation, and evaluation of algorithms designed to identify health status (DEVELOP-RCD). Our initial efforts involved conducting both a narrative review and a systematic review of published studies on the concepts and methodological issues related to algorithm development, validation, and evaluation. Subsequently, we conducted an empirical study on an algorithm for identifying sepsis. Based on these findings, we formulated specific workflow and recommendations for algorithm development, validation, and evaluation within the
guidance. Finally, the
guidance underwent independent review by a panel of 20 external experts who then convened a consensus meeting to finalize it.
RESULTS: A standardized workflow for algorithm development, validation, and evaluation was established. Guided by specific health status considerations, the workflow comprises four integrated steps: assessing an existing algorithm\'s suitability for the target health status; developing a new algorithm using recommended methods; validating the algorithm using prescribed performance measures; and evaluating the impact of the algorithm on study results. Additionally, 13 good practice recommendations were formulated with detailed explanations. Furthermore, a practical study on sepsis identification was included to demonstrate the application of this guidance.
CONCLUSIONS: The establishment of
guidance is intended to aid researchers and clinicians in the appropriate and accurate development and application of algorithms for identifying health status from RCD. This
guidance has the potential to enhance the credibility of findings from observational studies involving RCD.