PDF(Pubmed)
Abstract：
OBJECTIVE: To develop and validate machine learning (ML) models to predict high-flow nasal cannula (HFNC) failure in COVID-19, compare their performance to the respiratory rate-oxygenation (ROX) index, and evaluate model accuracy by self-reported race.
METHODS: Retrospective cohort study.
METHODS: Four Emory University Hospitals in Atlanta, GA.
METHODS: Adult patients hospitalized with COVID-19 between March 2020 and April 2022 who received HFNC therapy within 24 hours of ICU admission were included.
METHODS: None.
RESULTS: Four types of supervised ML models were developed for predicting HFNC failure (defined as intubation or death within 7 d of HFNC initiation), using routine clinical variables from the first 24 hours of ICU admission. Models were trained on the first 60% (n = 594) of admissions and validated on the latter 40% (n = 390) of admissions to simulate prospective implementation. Among 984 patients included, 317 patients (32.2%) developed HFNC failure. eXtreme Gradient Boosting (XGB) model had the highest area under the receiver-operator characteristic curve (AUROC) for predicting HFNC failure (0.707), and was the only model with significantly better performance than the ROX index (AUROC 0.616). XGB model had significantly worse performance in Black patients compared with White patients (AUROC 0.663 vs. 0.808, p = 0.02). Racial differences in the XGB model were reduced and no longer statistically significant when restricted to patients with nonmissing arterial blood gas data, and when XGB model was developed to predict mortality (rather than the composite outcome of failure, which could be influenced by biased clinical decisions for intubation).
CONCLUSIONS: Our XGB model had better discrimination for predicting HFNC failure in COVID-19 than the ROX index, but had racial differences in accuracy of predictions. Further studies are needed to understand and mitigate potential sources of biases in clinical ML models and to improve their equitability.
METHODS: Retrospective cohort study.
METHODS: Four Emory University Hospitals in Atlanta, GA.
METHODS: Adult patients hospitalized with COVID-19 between March 2020 and April 2022 who received HFNC therapy within 24 hours of ICU admission were included.
METHODS: None.
RESULTS: Four types of supervised ML models were developed for predicting HFNC failure (defined as intubation or death within 7 d of HFNC initiation), using routine clinical variables from the first 24 hours of ICU admission. Models were trained on the first 60% (n = 594) of admissions and validated on the latter 40% (n = 390) of admissions to simulate prospective implementation. Among 984 patients included, 317 patients (32.2%) developed HFNC failure. eXtreme Gradient Boosting (XGB) model had the highest area under the receiver-operator characteristic curve (AUROC) for predicting HFNC failure (0.707), and was the only model with significantly better performance than the ROX index (AUROC 0.616). XGB model had significantly worse performance in Black patients compared with White patients (AUROC 0.663 vs. 0.808, p = 0.02). Racial differences in the XGB model were reduced and no longer statistically significant when restricted to patients with nonmissing arterial blood gas data, and when XGB model was developed to predict mortality (rather than the composite outcome of failure, which could be influenced by biased clinical decisions for intubation).
CONCLUSIONS: Our XGB model had better discrimination for predicting HFNC failure in COVID-19 than the ROX index, but had racial differences in accuracy of predictions. Further studies are needed to understand and mitigate potential sources of biases in clinical ML models and to improve their equitability.
摘要：
目的：开发和验证机器学习（ML）模型以预测COVID-19的高流量鼻插管（HFNC）故障，并将其性能与呼吸频率氧合（ROX）指数进行比较，并通过自我报告的种族评估模型准确性。
方法：回顾性队列研究。
方法：亚特兰大埃默里大学四所医院，GA.
方法：纳入2020年3月至2022年4月期间因COVID-19住院、在入住ICU24小时内接受HFNC治疗的成年患者。
方法：无。
结果：开发了四种类型的监督ML模型来预测HFNC失败（定义为在HFNC开始后7天内插管或死亡），使用ICU入院前24小时的常规临床变量。模型在入院的前60％（n=594）进行了训练，并在入院的后40％（n=390）进行了验证，以模拟预期实施。在984名患者中，317例患者（32.2%）出现HFNC失败。极限梯度提升（XGB）模型在预测HFNC故障的接收器-操作者特征曲线（AUROC）下具有最高面积（0.707），并且是唯一性能明显优于ROX指数（AUROC0.616）的模型。与白人患者相比，XGB模型在黑人患者中的表现明显更差（AUROC0.663vs.0.808，p=0.02）。当仅限于动脉血气数据未缺失的患者时，XGB模型中的种族差异减少，不再具有统计学意义。当XGB模型被开发来预测死亡率(而不是失败的综合结果，这可能会受到有偏见的插管临床决策的影响)。
结论：我们的XGB模型对预测COVID-19HFNC失败的判别优于ROX指数，但是预测的准确性存在种族差异。需要进一步的研究来理解和减轻临床ML模型中潜在的偏见来源，并提高其公平性。
方法：回顾性队列研究。
方法：亚特兰大埃默里大学四所医院，GA.
方法：纳入2020年3月至2022年4月期间因COVID-19住院、在入住ICU24小时内接受HFNC治疗的成年患者。
方法：无。
结果：开发了四种类型的监督ML模型来预测HFNC失败（定义为在HFNC开始后7天内插管或死亡），使用ICU入院前24小时的常规临床变量。模型在入院的前60％（n=594）进行了训练，并在入院的后40％（n=390）进行了验证，以模拟预期实施。在984名患者中，317例患者（32.2%）出现HFNC失败。极限梯度提升（XGB）模型在预测HFNC故障的接收器-操作者特征曲线（AUROC）下具有最高面积（0.707），并且是唯一性能明显优于ROX指数（AUROC0.616）的模型。与白人患者相比，XGB模型在黑人患者中的表现明显更差（AUROC0.663vs.0.808，p=0.02）。当仅限于动脉血气数据未缺失的患者时，XGB模型中的种族差异减少，不再具有统计学意义。当XGB模型被开发来预测死亡率(而不是失败的综合结果，这可能会受到有偏见的插管临床决策的影响)。
结论：我们的XGB模型对预测COVID-19HFNC失败的判别优于ROX指数，但是预测的准确性存在种族差异。需要进一步的研究来理解和减轻临床ML模型中潜在的偏见来源，并提高其公平性。
