目的:内科医师等级对于住院医师选择很重要,但是评估者评分之间的不一致威胁到他们准确代表学生表现和感知公平的能力。文员等级委员会被推荐为最佳实践,但是它们促进准确性和公平性的机制并不确定。以前尚未研究过委员会可靠地评估和解释单个评估者的分级严格性的能力。
方法:这是对被认为是严格的教师完成的评估的回顾性分析,宽大处理,或由单一医学院的评分委员会成员组成的中立评分者。评估了教师评估,以评估个人技能的等级差异以及感知的严格性类别之间的最终等级建议。Logistic回归用于确定实际分配的评分是否根据感知的教师的评分严格类别而有所不同。
结果:“Easygraders”始终具有最高的概率授予高于平均水平的评级,而“高年级学生”获得高于平均水平的评级的概率始终最低,尽管这一发现仅在评估表上的8个问题中的2个达到统计学意义(P=.033和P=.001)。分配较高的最终建议成绩的几率遵循预期的模式(与“难”相比,“容易”和“中性”的几率更高,与“中性”相比,“容易”的“较高”),但未达到统计学意义。
结论:教师等级严格性的感知差异在实际中具有职务鉴定要素的依据。然而,被认为是“严格”或“宽松”的教师推荐的最终成绩没有差异。对“鹰”和“鸽子”的看法不仅是传说,而且可能不会对学生的最终成绩产生影响。持续的研究来描述“鹰鸽效应”对于评估当地的等级差异和授权当地教育领导纠正至关重要,但不要太正确,为了保持学生评价的公平性。
OBJECTIVE: Internal medicine clerkship grades are important for residency selection, but inconsistencies between evaluator ratings threaten their ability to accurately represent student performance and perceived fairness. Clerkship grading committees are recommended as best practice, but the mechanisms by which they promote accuracy and fairness are not certain. The ability of a committee to reliably assess and account for grading stringency of individual evaluators has not been previously studied.
METHODS: This is a retrospective analysis of evaluations completed by faculty considered to be stringent, lenient, or neutral graders by members of a grading committee of a single medical college. Faculty evaluations were assessed for differences in ratings on individual skills and recommendations for final grade between perceived stringency categories. Logistic regression was used to determine if actual assigned ratings varied based on perceived faculty\'s grading stringency category.
RESULTS: \"Easy graders\" consistently had the highest probability of awarding an above-average rating, and \"hard graders\" consistently had the lowest probability of awarding an above-average rating, though this finding only reached statistical significance only for 2 of 8 questions on the evaluation form (P = .033 and P = .001). Odds ratios of assigning a higher final suggested grade followed the expected pattern (higher for \"easy\" and \"neutral\" compared to \"hard,\" higher for \"easy\" compared to \"neutral\") but did not reach statistical significance.
CONCLUSIONS: Perceived differences in faculty grading stringency have basis in reality for clerkship evaluation elements. However, final grades recommended by faculty perceived as \"stringent\" or \"lenient\" did not differ. Perceptions of \"hawks\" and \"doves\" are not just lore but may not have implications for students\' final grades. Continued research to describe the \"hawk and dove effect\" will be crucial to enable assessment of local grading variation and empower local educational leadership to correct, but not overcorrect, for this effect to maintain fairness in student evaluations.