PDF(Pubmed)
Abstract：
BACKGROUND: Assessment of undergraduate students using assessment instruments in the clinical setting is known to be complex. The aim of this study was therefore to examine whether two different assessment instruments, containing learning objectives (LO`s) with similar content, results in similar assessments by the clinical supervisors and to explore clinical supervisors\' experiences of assessment regarding the two different assessment instruments.
METHODS: A mixed-methods approach was used. Four simulated care encounter scenarios were evaluated by 50 supervisors using two different assessment instruments. 28 follow-up interviews were conducted. Descriptive statistics and logistic binary regression were used for quantitative data analysis, along with qualitative thematic analysis of interview data.
RESULTS: While significant differences were observed within the assessment instruments, the differences were consistent between the two instruments, indicating that the quality of the assessment instruments were considered equivalent. Supervisors noted that the relationship between the students and supervisors could introduce subjectivity in the assessments and that working in groups of supervisors could be advantageous. In terms of formative assessments, the Likert scale was considered a useful tool for evaluating learning objectives. However, supervisors had different views on grading scales and the need for clear definitions. The supervisors concluded that a complicated assessment instrument led to limited very-day usage and did not facilitate formative feedback. Furthermore, supervisors discussed how their experiences influenced the use of the assessment instruments, which resulted in different descriptions of the experience. These differences led to a discussion of the need of supervisor teams to enhance the validity of assessments.
CONCLUSIONS: The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments. The quantitative data suggests that supervisors struggled with subjectivity, phrasing, and definitions of the LO´s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. To mitigate the subjectivity, supervisors suggested working in teams and conducting multiple assessments over time to increase assessment validity.
METHODS: A mixed-methods approach was used. Four simulated care encounter scenarios were evaluated by 50 supervisors using two different assessment instruments. 28 follow-up interviews were conducted. Descriptive statistics and logistic binary regression were used for quantitative data analysis, along with qualitative thematic analysis of interview data.
RESULTS: While significant differences were observed within the assessment instruments, the differences were consistent between the two instruments, indicating that the quality of the assessment instruments were considered equivalent. Supervisors noted that the relationship between the students and supervisors could introduce subjectivity in the assessments and that working in groups of supervisors could be advantageous. In terms of formative assessments, the Likert scale was considered a useful tool for evaluating learning objectives. However, supervisors had different views on grading scales and the need for clear definitions. The supervisors concluded that a complicated assessment instrument led to limited very-day usage and did not facilitate formative feedback. Furthermore, supervisors discussed how their experiences influenced the use of the assessment instruments, which resulted in different descriptions of the experience. These differences led to a discussion of the need of supervisor teams to enhance the validity of assessments.
CONCLUSIONS: The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments. The quantitative data suggests that supervisors struggled with subjectivity, phrasing, and definitions of the LO´s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. To mitigate the subjectivity, supervisors suggested working in teams and conducting multiple assessments over time to increase assessment validity.
摘要：
背景：在临床环境中使用评估工具对本科生进行评估是复杂的。因此，这项研究的目的是检查两种不同的评估工具，包含具有相似内容的学习目标(LO)，结果由临床主管进行类似的评估，并探索临床主管关于两种不同评估工具的评估经验。
方法：采用混合方法。50名主管使用两种不同的评估工具评估了四种模拟的护理遭遇情景。进行了28次后续访谈。描述性统计和逻辑二元回归用于定量数据分析，以及访谈数据的定性主题分析。
结果：虽然在评估工具中观察到显著差异，两种仪器之间的差异是一致的，表明评估工具的质量被认为是等同的。主管指出，学生和主管之间的关系可能会在评估中引入主观性，并且在主管小组中工作可能是有利的。在形成性评估方面，李克特量表被认为是评估学习目标的有用工具.然而,主管对分级标准和明确定义的必要性有不同的看法。监督员得出结论，复杂的评估工具导致了有限的日常使用，并且不利于形成性反馈。此外，监督员讨论了他们的经验如何影响评估工具的使用，这导致了对体验的不同描述。这些差异导致了对主管团队提高评估有效性的必要性的讨论。
结论：研究结果表明，使用两种不同的评估工具，通过/失败等级没有显着差异。定量数据表明，主管与主观性作斗争，短语,以及两种仪器中使用的LO和刻度的定义。这导致了耗时的任意评估，并导致日常评估的使用有限。为了减轻主观性，主管建议团队合作，并随着时间的推移进行多次评估，以提高评估的有效性。
方法：采用混合方法。50名主管使用两种不同的评估工具评估了四种模拟的护理遭遇情景。进行了28次后续访谈。描述性统计和逻辑二元回归用于定量数据分析，以及访谈数据的定性主题分析。
结果：虽然在评估工具中观察到显著差异，两种仪器之间的差异是一致的，表明评估工具的质量被认为是等同的。主管指出，学生和主管之间的关系可能会在评估中引入主观性，并且在主管小组中工作可能是有利的。在形成性评估方面，李克特量表被认为是评估学习目标的有用工具.然而,主管对分级标准和明确定义的必要性有不同的看法。监督员得出结论，复杂的评估工具导致了有限的日常使用，并且不利于形成性反馈。此外，监督员讨论了他们的经验如何影响评估工具的使用，这导致了对体验的不同描述。这些差异导致了对主管团队提高评估有效性的必要性的讨论。
结论：研究结果表明，使用两种不同的评估工具，通过/失败等级没有显着差异。定量数据表明，主管与主观性作斗争，短语,以及两种仪器中使用的LO和刻度的定义。这导致了耗时的任意评估，并导致日常评估的使用有限。为了减轻主观性，主管建议团队合作，并随着时间的推移进行多次评估，以提高评估的有效性。
