PDF(Pubmed)
Abstract：
BACKGROUND: Early warning score systems are widely used for identifying patients who are at the highest risk of deterioration to assist clinical decision-making. This could facilitate early intervention and consequently improve patient outcomes; for example, the National Early Warning Score (NEWS) system, which is recommended by the Royal College of Physicians in the United Kingdom, uses predefined alerting thresholds to assign scores to patients based on their vital signs. However, there is limited evidence of the reliability of such scores across patient cohorts in the United Arab Emirates.
OBJECTIVE: Our aim in this study was to propose a data-driven model that accurately predicts in-hospital deterioration in an inpatient cohort in the United Arab Emirates.
METHODS: We conducted a retrospective cohort study using a real-world data set that consisted of 16,901 unique patients associated with 26,073 inpatient emergency encounters and 951,591 observation sets collected between April 2015 and August 2021 at a large multispecialty hospital in Abu Dhabi, United Arab Emirates. The observation sets included routine measurements of heart rate, respiratory rate, systolic blood pressure, level of consciousness, temperature, and oxygen saturation, as well as whether the patient was receiving supplementary oxygen. We divided the data set of 16,901 unique patients into training, validation, and test sets consisting of 11,830 (70%; 18,319/26,073, 70.26% emergency encounters), 3397 (20.1%; 5206/26,073, 19.97% emergency encounters), and 1674 (9.9%; 2548/26,073, 9.77% emergency encounters) patients, respectively. We defined an adverse event as the occurrence of admission to the intensive care unit, mortality, or both if the patient was admitted to the intensive care unit first. On the basis of 7 routine vital signs measurements, we assessed the performance of the NEWS system in detecting deterioration within 24 hours using the area under the receiver operating characteristic curve (AUROC). We also developed and evaluated several machine learning models, including logistic regression, a gradient-boosting model, and a feed-forward neural network.
RESULTS: In a holdout test set of 2548 encounters with 95,755 observation sets, the NEWS system achieved an overall AUROC value of 0.682 (95% CI 0.673-0.690). In comparison, the best-performing machine learning models, which were the gradient-boosting model and the neural network, achieved AUROC values of 0.778 (95% CI 0.770-0.785) and 0.756 (95% CI 0.749-0.764), respectively. Our interpretability results highlight the importance of temperature and respiratory rate in predicting patient deterioration.
CONCLUSIONS: Although traditional early warning score systems are the dominant form of deterioration prediction models in clinical practice today, we strongly recommend the development and use of cohort-specific machine learning models as an alternative. This is especially important in external patient cohorts that were unseen during model development.
OBJECTIVE: Our aim in this study was to propose a data-driven model that accurately predicts in-hospital deterioration in an inpatient cohort in the United Arab Emirates.
METHODS: We conducted a retrospective cohort study using a real-world data set that consisted of 16,901 unique patients associated with 26,073 inpatient emergency encounters and 951,591 observation sets collected between April 2015 and August 2021 at a large multispecialty hospital in Abu Dhabi, United Arab Emirates. The observation sets included routine measurements of heart rate, respiratory rate, systolic blood pressure, level of consciousness, temperature, and oxygen saturation, as well as whether the patient was receiving supplementary oxygen. We divided the data set of 16,901 unique patients into training, validation, and test sets consisting of 11,830 (70%; 18,319/26,073, 70.26% emergency encounters), 3397 (20.1%; 5206/26,073, 19.97% emergency encounters), and 1674 (9.9%; 2548/26,073, 9.77% emergency encounters) patients, respectively. We defined an adverse event as the occurrence of admission to the intensive care unit, mortality, or both if the patient was admitted to the intensive care unit first. On the basis of 7 routine vital signs measurements, we assessed the performance of the NEWS system in detecting deterioration within 24 hours using the area under the receiver operating characteristic curve (AUROC). We also developed and evaluated several machine learning models, including logistic regression, a gradient-boosting model, and a feed-forward neural network.
RESULTS: In a holdout test set of 2548 encounters with 95,755 observation sets, the NEWS system achieved an overall AUROC value of 0.682 (95% CI 0.673-0.690). In comparison, the best-performing machine learning models, which were the gradient-boosting model and the neural network, achieved AUROC values of 0.778 (95% CI 0.770-0.785) and 0.756 (95% CI 0.749-0.764), respectively. Our interpretability results highlight the importance of temperature and respiratory rate in predicting patient deterioration.
CONCLUSIONS: Although traditional early warning score systems are the dominant form of deterioration prediction models in clinical practice today, we strongly recommend the development and use of cohort-specific machine learning models as an alternative. This is especially important in external patient cohorts that were unseen during model development.
摘要：
背景：早期预警评分系统广泛用于识别恶化风险最高的患者，以协助临床决策。这可以促进早期干预，从而改善患者预后；例如，国家预警评分（NEWS）系统，这是由英国皇家内科医学院推荐的，使用预定义的警报阈值根据患者的生命体征为其分配分数。然而,在阿拉伯联合酋长国的患者队列中,此类评分的可靠性证据有限.
目的：我们在这项研究中的目的是提出一种数据驱动模型，该模型可以准确预测阿拉伯联合酋长国住院队列中的住院恶化情况。
方法：我们使用真实世界数据集进行了一项回顾性队列研究，该数据集包括2015年4月至2021年8月在阿布扎比一家大型多专科医院收集的16,901名与26,073例住院急诊相关的独特患者和951,591个观察集。阿拉伯联合酋长国。观察集包括心率的常规测量，呼吸频率,收缩压,意识水平,温度,和氧饱和度,以及患者是否接受补充氧气。我们将16,901名独特患者的数据集分为培训，验证,和测试集包括11,830（70％；18,319/26,073，70.26％的紧急遭遇），3397（20.1%；5206/26,073，19.97%紧急遭遇），和1674（9.9％；2548/26,073，9.77％的紧急遭遇）患者，分别。我们将不良事件定义为重症监护病房的发生，死亡率,如果患者先被送进重症监护室，或者两者兼而有之。在7项常规生命体征测量的基础上，我们使用受试者工作特征曲线下面积(AUROC)评估了NEWS系统检测24小时内恶化的性能.我们还开发并评估了几种机器学习模型，包括逻辑回归，梯度提升模型，和前馈神经网络。
结果：在2548个遇到95,755个观察集的保持测试集中，新闻系统的总体AUROC值为0.682（95%CI0.673-0.690）。相比之下,性能最好的机器学习模型，梯度提升模型和神经网络，AUROC值为0.778（95%CI0.770-0.785）和0.756（95%CI0.749-0.764），分别。我们的可解释性结果强调了温度和呼吸频率在预测患者恶化中的重要性。
结论：尽管传统的早期预警评分系统是当今临床实践中恶化预测模型的主要形式，我们强烈建议开发和使用特定队列的机器学习模型作为替代方法.这在模型开发过程中看不见的外部患者队列中尤其重要。
目的：我们在这项研究中的目的是提出一种数据驱动模型，该模型可以准确预测阿拉伯联合酋长国住院队列中的住院恶化情况。
方法：我们使用真实世界数据集进行了一项回顾性队列研究，该数据集包括2015年4月至2021年8月在阿布扎比一家大型多专科医院收集的16,901名与26,073例住院急诊相关的独特患者和951,591个观察集。阿拉伯联合酋长国。观察集包括心率的常规测量，呼吸频率,收缩压,意识水平,温度,和氧饱和度,以及患者是否接受补充氧气。我们将16,901名独特患者的数据集分为培训，验证,和测试集包括11,830（70％；18,319/26,073，70.26％的紧急遭遇），3397（20.1%；5206/26,073，19.97%紧急遭遇），和1674（9.9％；2548/26,073，9.77％的紧急遭遇）患者，分别。我们将不良事件定义为重症监护病房的发生，死亡率,如果患者先被送进重症监护室，或者两者兼而有之。在7项常规生命体征测量的基础上，我们使用受试者工作特征曲线下面积(AUROC)评估了NEWS系统检测24小时内恶化的性能.我们还开发并评估了几种机器学习模型，包括逻辑回归，梯度提升模型，和前馈神经网络。
结果：在2548个遇到95,755个观察集的保持测试集中，新闻系统的总体AUROC值为0.682（95%CI0.673-0.690）。相比之下,性能最好的机器学习模型，梯度提升模型和神经网络，AUROC值为0.778（95%CI0.770-0.785）和0.756（95%CI0.749-0.764），分别。我们的可解释性结果强调了温度和呼吸频率在预测患者恶化中的重要性。
结论：尽管传统的早期预警评分系统是当今临床实践中恶化预测模型的主要形式，我们强烈建议开发和使用特定队列的机器学习模型作为替代方法.这在模型开发过程中看不见的外部患者队列中尤其重要。
