PDF(Pubmed)
Abstract：
OBJECTIVE: This study aimed to develop and evaluate an algorithm to reduce the chart review burden of improvement efforts by automatically labeling antibiotic selection as either guideline-concordant or -discordant based on electronic health record data for patients with community-acquired pneumonia (CAP).
METHODS: We developed a 3-part algorithm using structured and unstructured data to assess adherence to an institutional CAP clinical practice guideline. The algorithm was applied to retrospective data for patients seen with CAP from 2017 to 2019 at a tertiary children\'s hospital. Performance metrics included positive predictive value (precision), sensitivity (recall), and F1 score (harmonized mean), with macro-weighted averages. Two physician reviewers independently assigned \"actual\" labels based on manual chart review.
RESULTS: Of 1345 patients with CAP, 893 were included in the training cohort and 452 in the validation cohort. Overall, the model correctly labeled 435 of 452 (96%) patients. Of the 286 patients who met guideline inclusion criteria, 193 (68%) were labeled as having received guideline-concordant antibiotics, 48 (17%) were labeled as likely in a scenario in which deviation from the clinical practice guideline was appropriate, and 45 (16%) were given the final label of \"possibly discordant, needs review.\" The sensitivity was 0.96, the positive predictive value was 0.97, and the F1 was 0.96.
CONCLUSIONS: An automated algorithm that uses structured and unstructured electronic health record data can accurately assess the guideline concordance of antibiotic selection for CAP. This tool has the potential to improve the efficiency of improvement efforts by reducing the manual chart review needed for quality measurement.
METHODS: We developed a 3-part algorithm using structured and unstructured data to assess adherence to an institutional CAP clinical practice guideline. The algorithm was applied to retrospective data for patients seen with CAP from 2017 to 2019 at a tertiary children\'s hospital. Performance metrics included positive predictive value (precision), sensitivity (recall), and F1 score (harmonized mean), with macro-weighted averages. Two physician reviewers independently assigned \"actual\" labels based on manual chart review.
RESULTS: Of 1345 patients with CAP, 893 were included in the training cohort and 452 in the validation cohort. Overall, the model correctly labeled 435 of 452 (96%) patients. Of the 286 patients who met guideline inclusion criteria, 193 (68%) were labeled as having received guideline-concordant antibiotics, 48 (17%) were labeled as likely in a scenario in which deviation from the clinical practice guideline was appropriate, and 45 (16%) were given the final label of \"possibly discordant, needs review.\" The sensitivity was 0.96, the positive predictive value was 0.97, and the F1 was 0.96.
CONCLUSIONS: An automated algorithm that uses structured and unstructured electronic health record data can accurately assess the guideline concordance of antibiotic selection for CAP. This tool has the potential to improve the efficiency of improvement efforts by reducing the manual chart review needed for quality measurement.
摘要：
■本研究旨在开发和评估一种算法，通过根据社区获得性肺炎（CAP）患者的电子健康记录数据，自动将抗生素选择标记为指南一致或不一致，从而减轻改善工作的图表审查负担。
■我们使用结构化和非结构化数据开发了3部分算法，以评估对机构CAP临床实践指南的遵守情况。该算法应用于2017年至2019年在三级儿童医院就诊的CAP患者的回顾性数据。性能指标包括正预测值(精度)，敏感度(召回)，和F1得分(协调平均值)，宏观加权平均数。两名医师评审员根据手动图表评审独立分配“实际”标签。
■在1345例CAP患者中，893人包括在训练队列中，452人包括在验证队列中。总的来说,该模型正确标记了452例患者中的435例(96%).在286名符合指南纳入标准的患者中，193（68%）被标记为接受了指南一致的抗生素，在偏离临床实践指南的情况下，48（17％）被标记为可能，45人(16%)被贴上了“可能不和谐”的最终标签，需要审查。“敏感性为0.96，阳性预测值为0.97，F1为0.96。
■一种使用结构化和非结构化电子健康记录数据的自动化算法，可以准确地评估用于CAP的抗生素选择的指南一致性。该工具有可能通过减少质量测量所需的手动图表审查来提高改进工作的效率。
■我们使用结构化和非结构化数据开发了3部分算法，以评估对机构CAP临床实践指南的遵守情况。该算法应用于2017年至2019年在三级儿童医院就诊的CAP患者的回顾性数据。性能指标包括正预测值(精度)，敏感度(召回)，和F1得分(协调平均值)，宏观加权平均数。两名医师评审员根据手动图表评审独立分配“实际”标签。
■在1345例CAP患者中，893人包括在训练队列中，452人包括在验证队列中。总的来说,该模型正确标记了452例患者中的435例(96%).在286名符合指南纳入标准的患者中，193（68%）被标记为接受了指南一致的抗生素，在偏离临床实践指南的情况下，48（17％）被标记为可能，45人(16%)被贴上了“可能不和谐”的最终标签，需要审查。“敏感性为0.96，阳性预测值为0.97，F1为0.96。
■一种使用结构化和非结构化电子健康记录数据的自动化算法，可以准确地评估用于CAP的抗生素选择的指南一致性。该工具有可能通过减少质量测量所需的手动图表审查来提高改进工作的效率。
