PDF(Pubmed)
Abstract：
Introduction Whenever a department implements the evaluation of professionals, a reasonable operational goal is to request as few evaluations as possible. In anesthesiology, evaluations of anesthesiologists (by trainees) and nurse anesthetists (by anesthesiologists) with valid and psychometrically reliable scales have been made by requesting daily evaluations of the ratee\'s performance on the immediately preceding day. However, some trainees or nurse anesthetists are paired with the same anesthesiologist for multiple days of the same week. Multiple evaluations from the same rater during a given week may contribute little incremental information versus one evaluation from that rater for the week. We address whether daily evaluation requests could be adjusted adaptively to be made once per week, hopefully substantively reducing the number of evaluation requests. Methods Every day since 1 July 2013 at the studied department, anesthesia residents and fellows have been requested by email to evaluate anesthesiologists\' quality of supervision provided during the preceding day using the De Oliveira Filho supervision scale. Every day since 29 March 2015, the anesthesiologists have been requested by email to evaluate the work habits of the nurse anesthetists during the preceding day. Both types of evaluations were made for interactions throughout the workday together, not for individual cases. The criterion for an electronic request to be sent is that the pair worked together for at least one hour that day. The current study was performed using evaluations of anesthesiologists\' supervision and nurse anesthetists\' work habits through 30 June 2023. Results If every evaluation request were completed by trainees on the same day it was requested, trainees would have received 13.5% fewer requests to evaluate anesthesiologists (9367/69,420), the maximum possible reduction. If anesthesiologists were to do the same for their evaluations of nurse anesthetists, the maximum possible reduction would be 7.1% fewer requests (4794/67,274). However, because most evaluations were completed after the day of the request (71%, 96,451/136,694), there would be fewer requests only if the evaluation were completed before or on the day of the next pairing. Consequently, in actual practice, there would have been only 2.4% fewer evaluation requests to trainees and 1.5% fewer to anesthesiologists, both decreases being significantly less than 5% (both adjusted P <0.0001). Among the trainees\' evaluations of faculty anesthesiologists, there were 1.4% with very low scores, specifically, a mean score of less than three out of four (708/41,778). Using Bernoulli cumulative sum (CUSUM) among successive evaluations, 72 flags were raised over the 10 years. Among those, there were 36% with more than one rater giving an exceptionally low score during the same week (26/72). There were 97% (70/72) with at least one rater contributing more than one score to the recent cumulative sum. Conclusion Conceptually, evaluation requests could be skipped if a rater has already evaluated the ratee that week during an earlier day working together. Our results show that the opportunity for reductions in evaluation requests is significantly less than 5%. There may also be impaired monitoring for the detection of sudden major decreases in ratee performance. Thus, the simpler strategy of requesting evaluations daily after working together is warranted.
摘要：
引言每当一个部门实施对专业人员的评价时，合理的操作目标是要求尽可能少的评估。在麻醉学中，使用有效且心理测量可靠的量表对麻醉医师(由受训者)和护士麻醉医师(由麻醉医师)进行评估,并要求在前一天对费率表现进行每日评估.然而,一些受训者或护士麻醉师与同一位麻醉师在同一周中的多天配对。在给定的一周中，来自同一评估者的多次评估可能贡献很少的增量信息，而该评估者在一周中的一次评估。我们解决是否可以自适应地调整每日评估请求以每周一次，希望能大幅减少评估请求的数量。方法自2013年7月1日起每天在研究科,已通过电子邮件要求麻醉住院医师和研究员使用DeOliveiraFilho监督量表评估前一天提供的麻醉医师监督质量。自2015年3月29日以来，每天都通过电子邮件要求麻醉医师评估前一天护士麻醉医师的工作习惯。这两种类型的评估都是在整个工作日一起进行的，不是个别情况。发送电子请求的标准是这对夫妇当天一起工作至少一个小时。目前的研究是通过对麻醉师的监督和护士麻醉师的工作习惯进行评估，直到2023年6月30日。结果如果每个评估请求都是由学员在要求的同一天完成的，学员收到的评估麻醉师的请求将减少13.5%(9367/69,420)，最大可能的减少。如果麻醉师对护士麻醉师的评估也是如此，最大可能的减少将减少7.1%的请求(4794/67,274)。然而,因为大多数评估是在请求当天完成的(71%，96,451/136,694)，只有在下一次配对之前或当天完成评估，请求才会减少。因此,在实际的实践中,对受训者的评估请求只会减少2.4%，对麻醉师的评估请求只会减少1.5%，两者的降幅均显著低于5%(均调整后P<0.0001)。在受训人员对麻醉师的评估中，有1.4%的分数非常低，具体来说,平均得分低于四分之三(708/41,778)。在连续评估中使用伯努利累积和(CUSUM)，在过去的十年中，升起了72面旗帜。其中,在同一周内（26/72），有36％的评估者得分异常低。有97％（70/72），至少有一个评分者对最近的累计总和贡献了一个以上的分数。结论从概念上讲，如果评估者已经在较早的一天中一起评估了该周的费率，则可以跳过评估请求。我们的结果表明，减少评估请求的机会明显低于5%。对于速率性能突然大幅下降的检测，也可能存在受损的监测。因此，更简单的策略是在一起工作后每天要求进行评估。
