背景:人工智能中偏见的存在引起了越来越多的关注,随着算法性能的不平等在刑事司法领域暴露出来,教育,和福利服务。在医疗保健方面,不同人口群体算法的不公平表现可能会扩大健康不平等。
目标:这里,我们识别和表征心脏病算法中的偏差,专门研究心力衰竭管理中使用的算法。
方法:阶段1涉及PubMed和WebofScience的文献检索与心脏机器学习(ML)算法相关的关键术语。评估了构建ML模型以预测心脏病的论文,以关注模型性能中的人口统计学偏差,并为我们的调查保留了开源数据集。确定了两个开源数据集:(1)加州大学欧文分校心力衰竭数据集和(2)加州大学欧文分校冠状动脉疾病数据集。我们复制了这些数据集的现有算法,测试了他们在算法性能上的性别偏见,并评估了一系列补救技术在减少不平等方面的功效。特别注意假阴性率(FNR),由于诊断不足和错过治疗机会的临床意义。
结果:在第一阶段,我们的文献检索返回127篇论文,有60篇达到了全面审查的标准,只有3篇论文强调了算法性能的性别差异。在报道性的报纸上,在数据集中,女性患者的代表性始终偏低.没有论文调查种族或民族差异。在第二阶段,我们再现了文献中报道的算法,数据集1的平均准确度为84.24%(SD3.51%),数据集2的平均准确度为85.72%(SD1.75%)(随机森林模型)。对于数据集1,在16个实验中的13个中,女性患者的FNR明显更高。达到阈值有统计学意义(-17.81%~-3.37%;P<.05)。在16个实验中的13个实验中,男性患者的假阳性率差异较小(-0.48%至9.77%;P<0.05)。我们观察到男性患者的疾病预测过高(假阳性率较高),女性患者的疾病预测过低(FNR较高)。特征重要性的性别差异表明,特征选择需要根据人口统计进行调整。
结论:我们的研究揭示了心脏ML研究的显著差距,强调女性患者算法的性能不佳在已发表的文献中被忽视了。我们的研究量化了算法性能中的性别差异,并探讨了偏见的几种来源。我们在用于训练算法的数据集中发现女性患者的代表性不足,识别出模型错误率中的性别偏见,并证明了一系列补救技术无法解决存在的不平等问题。
BACKGROUND: The presence of bias in artificial intelligence has garnered increased attention, with inequities in algorithmic performance being exposed across the fields of criminal justice, education, and welfare services. In health care, the inequitable performance of algorithms across demographic groups may widen health inequalities.
OBJECTIVE: Here, we identify and characterize bias in cardiology algorithms, looking specifically at algorithms used in the management of heart failure.
METHODS: Stage 1 involved a literature search of PubMed and Web of Science for key terms relating to cardiac machine learning (ML) algorithms. Papers that built ML models to predict cardiac disease were evaluated for their focus on demographic bias in model performance, and open-source data sets were retained for our investigation. Two open-source data sets were identified: (1) the University of California Irvine Heart Failure data set and (2) the University of California Irvine Coronary Artery Disease data set. We reproduced existing algorithms that have been reported for these data sets, tested them for sex biases in algorithm performance, and assessed a range of remediation techniques for their efficacy in reducing inequities. Particular attention was paid to the false negative rate (FNR), due to the clinical significance of underdiagnosis and missed opportunities for treatment.
RESULTS: In stage 1, our literature search returned 127 papers, with 60 meeting the criteria for a full review and only 3 papers highlighting sex differences in algorithm performance. In the papers that reported sex, there was a consistent underrepresentation of female patients in the data sets. No papers investigated racial or ethnic differences. In stage 2, we reproduced algorithms reported in the literature, achieving mean accuracies of 84.24% (SD 3.51%) for data set 1 and 85.72% (SD 1.75%) for data set 2 (random forest models). For data set 1, the FNR was significantly higher for female patients in 13 out of 16 experiments, meeting the threshold of statistical significance (-17.81% to -3.37%; P<.05). A smaller disparity in the false positive rate was significant for male patients in 13 out of 16 experiments (-0.48% to +9.77%; P<.05). We observed an overprediction of disease for male patients (higher false positive rate) and an underprediction of disease for female patients (higher FNR). Sex differences in feature importance suggest that feature selection needs to be demographically tailored.
CONCLUSIONS: Our research exposes a significant gap in cardiac ML research, highlighting that the underperformance of algorithms for female patients has been overlooked in the published literature. Our study quantifies sex disparities in algorithmic performance and explores several sources of bias. We found an underrepresentation of female patients in the data sets used to train algorithms, identified sex biases in model error rates, and demonstrated that a series of remediation techniques were unable to address the inequities present.