RESULTS: We find that, compared with the unconstrained model, group-recalibration improves calibration at one of the relevant thresholds for each group, but exacerbates differences in false positive and false negative rates between groups. An equalised odds constraint, meant to equalise error rates across groups, does so by miscalibrating the model overall and at relevant decision thresholds.
CONCLUSIONS: Hence, because of induced miscalibration, decisions guided by risk estimators learned with an equalised odds fairness constraint are not concordant with existing guidelines. Conversely, recalibrating the model separately for each group can increase guideline compatibility, while increasing intergroup differences in error rates. As such, comparisons of error rates across groups can be misleading when guidelines recommend treating at fixed decision thresholds.
CONCLUSIONS: The illustrated tradeoffs between satisfying a fairness criterion and retaining guideline compatibility underscore the need to evaluate models in the context of downstream interventions.
结果:我们发现,与无约束模型相比,组重新校准改进了每个组的相关阈值之一的校准,但加剧了组间假阳性和假阴性率的差异。均衡赔率约束,旨在均衡各组的错误率,这样做是通过错误校准整个模型和相关的决策阈值。
结论:因此,由于诱发的校准错误,由风险估计器指导的决策与均衡的赔率公平约束不一致的现有准则。相反,为每个组分别重新校准模型可以增加指南兼容性,同时增加了错误率的组间差异。因此,当指南建议以固定的决策阈值进行治疗时,组间错误率的比较可能会产生误导.
结论:说明的满足公平性标准和保持指南兼容性之间的权衡强调了在下游干预背景下评估模型的必要性。