■在纠正医疗数据中的“类不平衡”问题时,重采样对分类器算法的影响尚不清楚.我们研究了分类器和重采样率的几种组合对性能的影响。
■在7个重新采样的数据集上训练了多种分类算法:没有校正,随机欠采样,合成少数过采样技术(SMOTE)的4个比率,和随机过采样与自适应合成算法(ADASYN)。以曲线下面积(AUC)评估性能,精度,召回,Brier得分,和校准指标。介绍了先前入院的泌尿外科患者30天非计划再入院的预测模型的案例研究。
■对于大多数算法,使用重采样数据显示AUC和精度显着增加,范围从0.74(CI:0.69-0.79)到0.93(CI:0.92-0.94),和0.35(CI:0.12-0.58)至0.86(CI:0.81-0.92)。所有分类算法都显示出召回率显着增加,并且在失真校准高估阳性的情况下,Brier得分显着降低。
■不平衡校正导致整体性能提高,但校准不佳的模型。由于强大的辨别性能,仍然可以有临床效用,特别是在仅预测低风险和高风险病例时,在临床上更具相关性。
■重采样数据提高了分类算法的性能,然而却高估了积极的预测。根据我们案例研究的结果,周到的临床预测任务的预定义可以指导在未来的研究中使用重采样技术,旨在改善临床决策支持工具。
UNASSIGNED: When correcting for the \"class imbalance\" problem in medical data, the effects of resampling applied on classifier algorithms remain unclear. We examined the effect on performance over several combinations of classifiers and resampling ratios.
UNASSIGNED: Multiple classification algorithms were trained on 7 resampled datasets: no correction, random undersampling, 4 ratios of Synthetic Minority Oversampling Technique (SMOTE), and random oversampling with the Adaptive Synthetic algorithm (
ADASYN). Performance was evaluated in Area Under the Curve (AUC), precision, recall, Brier score, and calibration metrics. A case study on prediction modeling for 30-day unplanned readmissions in previously admitted Urology patients was presented.
UNASSIGNED: For most algorithms, using resampled data showed a significant increase in AUC and precision, ranging from 0.74 (CI: 0.69-0.79) to 0.93 (CI: 0.92-0.94), and 0.35 (CI: 0.12-0.58) to 0.86 (CI: 0.81-0.92) respectively. All classification algorithms showed significant increases in recall, and significant decreases in Brier score with distorted calibration overestimating positives.
UNASSIGNED: Imbalance correction resulted in an overall improved performance, yet poorly calibrated models. There can still be clinical utility due to a strong discriminating performance, specifically when predicting only low and high risk cases is clinically more relevant.
UNASSIGNED: Resampling data resulted in increased performances in classification algorithms, yet produced an overestimation of positive predictions. Based on the findings from our case study, a thoughtful predefinition of the clinical prediction task may guide the use of resampling techniques in future studies aiming to improve clinical decision support tools.