PDF(Pubmed)
Abstract：
Background Standardized letters of evaluation (SLOE) are becoming more widely incorporated into the residency application process to make the letter of recommendation, an already critical component in a residency application packet, more objective. However, it is not currently known if the reviewers of these letters share consensus regarding the strength of an applicant determined by their SLOE. Objective We measured the level of faculty agreement regarding applicant competitiveness as determined by SLOEs and the ability of 2 algorithms to predict faculty consensus rankings. Methods Using data from the 2021-2022 Match cycle from the Council of Residency Directors in Emergency Medicine SLOE Database as a blueprint, authors created 50 fictional SLOEs representative of the national data. Seven faculty then rated these SLOEs in order of applicant competitiveness, defined as suggested rank position. Consensus was evaluated using cutoffs established a priori, and 2 prediction models, a point-based system and a linear regression model, were tested to determine their ability to predict consensus rankings. Results There was strong faculty consensus regarding the interpretation of SLOEs. Within narrow windows of agreement, faculty demonstrated similar ranking patterns with 83% and 93% agreement for \"close\" and \"loose\" agreement, respectively. Predictive models yielded a strong correlation with the consensus ranking (point-based system r=0.97, linear regression r=0.97). Conclusions Faculty displayed strong consensus regarding the competitiveness of applicants via SLOEs, adding further support to the use of SLOEs for selection and advising. Two models predicted consensus competitiveness rankings with a high degree of accuracy.
摘要：
背景技术标准化评估信(SLOE)正越来越广泛地纳入住院医师申请流程，以制作推荐信，驻留应用程序数据包中已经很关键的组件，更客观。然而,目前尚不清楚这些信件的审稿人是否就其SLOE确定的申请人的实力达成共识。目的我们测量了由SLOE确定的关于申请人竞争力的教师协议水平以及2种算法预测教师共识排名的能力。方法使用来自急诊医学SLOE数据库住院医师理事会的2021-2022匹配周期的数据作为蓝图，作者创建了50个代表国家数据的虚构SLOE。然后，七名教师按照申请人竞争力的顺序对这些SLOE进行了评级，定义为建议的排名位置。共识是使用先验建立的截止值进行评估的，和2个预测模型，基于点的系统和线性回归模型，进行了测试，以确定他们预测共识排名的能力。结果教师对SLOEs的解释有很强的共识。在狭窄的协议窗口内，教师表现出类似的排名模式，83%和93%的“接近”和“松散”协议，分别。预测模型与共识排名具有很强的相关性（基于点的系统r=0.97，线性回归r=0.97）。结论教师通过SLOE对申请人的竞争力表现出强烈的共识，进一步支持使用SLOE进行选择和建议。两个模型预测共识竞争力排名具有很高的准确性。
