UNASSIGNED: Simulations are used to improve professional practice across multiple health professions; however, the application of simulations in pharmacy curricula has been limited by a lack of evidence.
UNASSIGNED: To delineate the competencies of pharmacy residents needed to assess their progress while participating in a high-fidelity simulation training program and to create a reliable assessment tool based on these competencies.
UNASSIGNED: A literature review was conducted, followed by application of a modified Delphi method. An assessment tool based on the results of these activities was drafted (in French). A second modified Delphi process was carried out to assess the reliability of the tool, and the tool was modified accordingly.
UNASSIGNED: The literature search yielded a total of 2670 articles, of which 289 were included for analysis. The first modified Delphi process involved 19 experts in the first round and 10 experts in the second round. The Cronbach α was 0.866 (95% confidence interval [CI] 0.713-0.960), indicating good reliability. A total of 7 competencies were retained for inclusion in the SIMPHARM assessment tool: professionalism, leadership, teamwork, communication, critical thinking, preparation and packaging of medications, and pharmaceutical calculations. The second modified Delphi process involved 9 experts in the first round and 11 experts in the second round. The final Cronbach α was 0.877 (95% CI 0.741-0.960).
UNASSIGNED: To the authors\' knowledge, this was one of the largest studies exploring pharmacy competencies in the context of simulations. This work yielded a reliable framework for the assessment of pharmacy residents\' competencies. This assessment tool may help evaluators in assessing the competencies of pharmacy trainees after simulation training.
UNASSIGNED: Les simulations sont utilisées dans le but d’améliorer la pratique professionnelle de plusieurs professions de la santé; cependant, le manque de données probantes limite leur application dans les programmes d’enseignement en pharmacie.
UNASSIGNED: Décrire les compétences nécessaires pour évaluer le progrès des résidents en pharmacie qui participent à une formation basée sur un programme de simulation haute-fidélité; et mettre au point un outil d’évaluation fiable qui se base sur ces compétences.
UNASSIGNED: Une revue de littérature a été effectuée, avant d’appliquer une méthode Delphi modifiée et de faire l’ébauche d’un outil d’évaluation (en français) basé sur les résultats de ces activités. Une deuxième méthode Delphi modifiée a été appliquée pour évaluer la fiabilité de l’outil et celui-ci a fait l’objet de modifications en conséquence.
UNASSIGNED: La revue de littérature a donné un total de 2670 articles, parmi lesquels 289 ont été inclus à des fins d’analyse. La première méthode Delphi modifiée a impliqué 19 experts pour la première phase, et 10 pour la seconde. Le coefficient alpha de Cronbach était de 0,866 (intervalle de confiance [IC] à 95 % 0,713–0,960), indiquant une bonne fiabilité. Au total, 7 compétences à inclure dans l’outil d’évaluation ont été retenues: professionnalisme, leadership, travail d’équipe, communication, pensée critique, préparation et conditionnement des médicaments, et calculs pharmaceutiques. La deuxième méthode Delphi modifiée impliquait quant à elle 9 experts pour la première phase et 11 pour la seconde. Le coefficient alpha de Cronbach final était de 0,877 (IC à 95 % 0,741–0,960).
UNASSIGNED: À la connaissance des auteurs, cette étude était l’une des plus importantes se penchant sur les compétences en pharmacie dans le contexte des simulations. Ce travail a donné lieu à un cadre de référence fiable pour évaluer les compétences des résidents en pharmacie. Cet outil d’évaluation pourrait aider les évaluateurs à évaluer les compétences des stagiaires à la suite d’une formation par simulation.

BACKGROUND: Immersive virtual reality (IVR) is a niche technology rising in popularity in nursing education. Although there is an abundance of evidence to demonstrate the effect of virtual reality (VR) on desired learning outcomes, this evidence is limited to technical or procedural skills or managing a single patient with clinical problems. Nontechnical skills (NTS), such as communication, decision-making, teamwork, situation awareness, and managerial skills, have not been explored using IVR technology.
OBJECTIVE: This study aimed to (1) investigate the potential efficacy of the IVR system virtual reality hospital (VR-Hospital, or VR-Hosp), a single-user game we developed, on nursing students\' NTS, sense of presence in the virtual clinical environment, and satisfaction and self-confidence in learning; (2) identify variables that predict NTS; and (3) explore students\' experience in using VR-Hosp.
METHODS: A multimethods design with a quantitative and qualitative approach was adopted. Participants were provided with VR-Hosp with 3 scenarios in training. VR-Hosp adopted a multibed, multipatient, multitask approach and was embedded with various clinical situations. Learning outcomes were measured after the training, followed by group interviews.
RESULTS: In total, 202 students joined the study. Results revealed high levels of satisfaction and self-confidence in learning. Significant achievement in NTS was perceived by the students. The levels of satisfaction and self-confidence in learning and the involvement and sensory fidelity domains in the sense of presence were positive predictors of NTS.
CONCLUSIONS: The promising results offer a basis for designing IVR activities for nursing education. Further investigations are imperative to determine the impact of IVR technology on learning outcomes in clinical practice.

BACKGROUND: Situational judgment tests (SJT) have gained popularity as a standardized assessment of nontechnical competencies for applicants to medical school and residency. SJT formats range from rating the effectiveness of potential response options to solely open response. We investigated differences in test-taking patterns between responders and nonresponders to optional open response SJT questions during the application process.
METHODS: This was a prospective multi-institutional study of general surgery applicants to seven residency programs. Applicants completed a 32-item SJT designed to measure ten core competencies: adaptability, attention to detail, communication, dependability, feedback receptivity, integrity, professionalism, resilience, self-directed learning, and team orientation. Each SJT item included an optional, nonscored, open response space for applicants to provide a behavioral response if they desired. Trends in applicant gender, race, ethnicity, medical school ranking, and USMLE scores were examined between the responder versus nonresponder group.
RESULTS: In total, 1491 general surgery applicants were invited to complete the surgery-specific SJT. Of these, 1454 (97.5%) candidates completed the assessment and 1177 (78.9%) provided additional responses to at least one of the 32 SJT scenario sets. There were no differences in overall SJT performance, USMLE scores (Step 1: 235, SD 14, Step 2: 250, SD 11), race and/or ethnicity between the responder and nonresponder groups. Responders were more likely to be from a top 25 medical school (p < 0.05) compared to the nonresponder group. Among applicants who completed any open response questions, women completed a significantly higher number of questions compared to men (7.21 vs 6.07, p = 0.003). The number of open responses provided correlated with higher scores on SJT items measuring dependability (r = 0.07, p = 0.007).
CONCLUSIONS: SJT design and format has the potential to impact test-taker response patterns. SJT developers and adopters should ensure test format and design have no unintended consequences prior to implementation.

Non-Technical Skills (NTS) of medical teams are currently measured using subjective and resource-intensive ratings given by experts. This study explores if objective NTS assessment approaches with eye-tracking and audio sensors can measure teamwork and communication skills in surgery. Eight surgeons participated in a simulated two-phase surgical scenario developed to assess their NTS. Sensor-based audio, eye tracking and video data were collected and analyzed along with rating from the NOTSS scale. Different levels of communication were detected by the sensor data during the two phases of the simulated surgery. Sensor data detected leadership qualities among surgeons based on speech metrics, and eye tracking offered additional evidence about gaze patterns related to NTS. This objective approach to NTS measurement captured differences in communication in greater detail as opposed to a single collective rating obtained using current assessment tools.

OBJECTIVE: A paucity of formal leadership training programs exists for residents, and outcomes of those are limited in reporting. Based on a robust needs assessment, our program created a longitudinal cohort curriculum, Future Surgical Leaders, for residents and fellows of all levels to provide training in nontechnical skills. Our objective was to evaluate surgical resident short-term outcomes and satisfaction with the Future Surgical Leaders (FSL) curriculum.
METHODS: Participants were sent a brief survey after each session of the curriculum from October 2020 to February 2022. The data was compiled after seventeen months of delivery. Likert Scale responses and text comments were analyzed with a 2-sample t-test and 2-way analysis of variance.
METHODS: Academic tertiary institution.
METHODS: General surgery residents.
RESULTS: Survey response rate from 54 sessions among all postgraduate year levels was 73%. Overall, 96% of residents/fellows either \"agreed\" or \"strongly agreed\" that the topics of the FSL curriculum were important to learn during surgical training. Only 24% of learners knew \"a lot\" or \"a great deal\" about the topics prior to the session which rose to 73% afterwards (p < 0.01). Each postgraduate year class showed statistically significant increase in knowledge. About 80% of learners wanted to investigate these topics further. Open comment questions identified themes requesting delivery of specific sessions earlier in residency training and positive overall attitudes toward the FSL curriculum.
CONCLUSIONS: FSL is a satisfactory means of teaching leadership skills to surgical residents. Residents recognize the need to develop leadership skills prior to entering practice and want to learn more. The FSL curriculum may be considered for application at other surgical training programs.

BACKGROUND: The aim of our cross-sectional study was to evaluate the current situation and curriculum of nontechnical skills (NTS) training in the undergraduate education of health care professionals in Hungary.
METHODS: All institutes with relevant NTS training in Hungarian faculties of medicine and faculties of health sciences were asked to fill out a 19-item questionnaire. Descriptive statistics were performed, and the characteristics of NTS teaching and non-NTS teaching institutes were compared. The independent predictors of teaching NTS in a particular institute were identified with multiple logistic regression.
RESULTS: Seventy-seven institutes responded (52% response rate), of which 66% trained NTS. The most frequent method of NTS training is talking about them during a practice or lecture, and less than half of NTS respondents use simulation. The most frequent cause of not teaching NTS is a lack of human or technical resources. The type of faculty (p = 0.025), academic year (p = 0.001), field of medicine (p = 0.025), and importance of teamwork (p = 0.021) differed between NTS and noNTS institutes. Teaching students in academic year two represented the only independent predictor of NTS education (p = 0.012).
CONCLUSIONS: Our findings show that the undergraduate curriculum of Hungarian universities includes some type of NTS education; however, this education requires further development.

BACKGROUND: The association between team performance and patient care was an immense boost for team-based education in health care. Behavioural skills are an important focus in these sessions, often provided via a mannikin-based immersive simulation experience in a (near) authentic setting. Observation of these skills by the facilitator(s) is paramount for facilitated feedback with the team. Despite the acknowledgement that trained facilitators are important for optimal learning, insight into this observation process by facilitators is limited.
OBJECTIVE: What are the self-reported current practices and difficulties regarding the observation of behavioural skills amongst facilitators during team training and how have they been trained to observe behavioural skills?
METHODS: This cross-sectional study used a pilot-tested, content-validated, multi-linguistic online survey within Europe, distributed through a non-discriminative snowball sampling method. Inclusion was limited to facilitators observing behavioural skills within a medical team setting.
RESULTS: A total of 175 persons filled in the questionnaire. All aspects of behavioural skill were perceived as very important to observe. The self-perceived difficulty of the behavioural skill aspects ranged from slightly to moderately difficult. Qualitative analysis revealed three major themes elaborating on this perceived difficulty: (1) not everything can be observed, (2) not everything is observed and (3) interpretation of observed behavioural skills is difficult. Additionally, the number of team members health care facilitators have to observe, outnumbers their self-reported maximum. Strategies and tools used to facilitate their observation were a blank notepad, co-observers and predefined learning goals. The majority of facilitators acquired observational skills through self-study and personal experience and/or observing peers. Co-observation with either peers or experts was regarded as most learn some for their expertise development. Overall, participants perceived themselves as moderately competent in the observation of behavioural skills during team training.
CONCLUSIONS: Observation of behavioural skills by facilitators in health care remains a complex and challenging task. Facilitators\' limitations with respect to attention, focus and (in)ability to perform concomitant tasks, need to be acknowledged. Although strategies and tools can help to facilitate the observation process, they all have their limitations and are used in different ways.

BACKGROUND: The NorMS-NTS tool is an assessment tool for assessing Norwegian medical students\' non-technical skills (NTS). The NorMS-NTS was designed to provide student feedback, training evaluations, and skill-level comparisons among students at different study sites. Rather than requiring extensive rater training, the tool should capably suit the needs of busy doctors as near-peer educators. The aim of this study was to examine the usability and preliminary assess validity of the NorMS-NTS tool when used by novice raters.
METHODS: This study focused on the usability of the assessment tool and its internal structure. Three raters used the NorMS-NTS tool to individually rate the team leader, a medical student, in 20 video-recorded multi-professional simulation-based team trainings. Based on these ratings, we examined the tools\' internal structure by calculating the intraclass correlation coefficient (ICC) (version 3.1) interrater reliability, internal consistency, and observability. After the rating process was completed, the raters answered a questionnaire about the tool\'s usability.
RESULTS: The ICC agreement and the sum of the overall global scores for all raters were fair: ICC (3,1) = 0.53. The correlation coefficients for the pooled raters were in the range of 0.77-0.91. Cronbach\'s alpha for elements, categories and global score were mostly above 0.90. The observability was high (95%-100%). All the raters found the tool easy to use, none of the elements were redundant, and the written instructions were helpful. The raters also found the tool easier to use once they had acclimated to it. All the raters stated that they could use the tool for both training and teaching.
CONCLUSIONS: The observed ICC agreement was 0.08 below the suggested ICC level for formative assessment (above 0.60). However, we know that the suggestion is based on the average ICC, which is always higher than a single-measure ICC. There are currently no suggested levels for single-measure ICC, but other validated NTS tools have single-measure ICC in the same range. We consider NorMS-NTS as a usable tool for formative assessment of Norwegian medical students\' non-technical skills during multi-professional team training by raters who are new to the tool. It is necessary to further examine validity and the consequences of the tool to fully validate it for formative assessments.

Proficiency in nontechnical skills (NTS) contributes to reduction in critical safety incidents and improvement in patient safety outcomes. Despite evidence demonstrating the importance of NTS in patient safety, there remains limited NTS specific curricula and formal teaching in Canadian surgical programs. We propose a three-stage longitudinal approach to education surrounding NTS using the Nontechnical skills for surgeons (NOTSS) framework.

Assessment tools for grading technical and nontechnical skills, such as operative technique and professionalism, are well established in general surgery. Less is known regarding the application of these tools in plastic surgery training. This study is a comparative review of the most prevalent assessment tools and rubrics utilized in general and plastic surgery. Two parallel systematic reviews of the literature utilizing PubMed and Cochrane were conducted for articles published between 1990 and 2022. Searches used Boolean operators specific to assessment tools in general and plastic surgery. Fourteen studies met the inclusion criteria for general surgery assessment tools, and 21 studies were included for plastic surgery assessment tools. Seven studies (50%) evaluated technical skills in general surgery, whereas 15 studies (71%) assessed technical skills in plastic surgery with commonality found in the evaluation of principles, such as tissue and instrument handling and operative flow. Task-specific evaluation tools were described for both general and plastic surgeries. Five studies evaluated nontechnical skills, such as communication and leadership in general surgery, whereas no plastic surgery studies solely examined nontechnical assessment tools. Our literature review demonstrates that standardized skill assessments in plastic surgery are lacking compared with those available in general surgery. Plastic surgery programs should consider implementing competency-based assessment tools in surgical coaching and training for technical and nontechnical skills. More research is necessary in plastic surgery to optimize the evaluation of nontechnical skills.