Simulation-Based Assessment to Evaluate Cognitive Performance in an Anesthesiology Residency Program

摘要

What was known Traditional assessment is not suited for measuring skills in uncertain situations that require higher-order cognitive processing.;What is new Simulation-based assessment differentiated between higher-order (cognitive) and lower-order (basic and technical) skills in relatively experienced anesthesiology residents.;Limitations Single-institution study and limited sample reduce generalizability.;Bottom line Gaps in anesthesia nontechnical skills (ANTS) can be addressed at the level of the individual trainees or, for recurring themes, should inform curriculum development.;Editor's Note: The online version of this article contains a table of ranked cognitive errors, details for how the feasibility and acceptability of the simulation assessment method were established, checklist items and clinical scenarios used in the study, and item descriptions and performance grading.;Introduction Applying learning theories1–3 and competency assessment4–6 of nontechnical and technical skills7,8 cannot be accomplished using only traditional examinations,9–12 including objective structured clinical examinations (OSCEs) and multiple-choice questions. These examinations fail to capture the uncertainty that will be encountered in some clinical scenarios. Problem solving in the operating room requires knowledge and experience.10 Current evaluations (including simulation-based assessments) typically measure basic knowledge and performance rather than competency in the complex tasks of acute care.13 This is why it is important to develop better methods to measure acute care clinical performance. Simulation could be used to measure advanced nontechnical cognitive diagnostic and therapeutic management skills and the ability to integrate knowledge, judgment, communication, and teamwork into the simulated practice setting. In this article, we follow the definition of anesthesia nontechnical skills (ANTS).14–18 These ANTS include task management, teamwork, situation awareness, and decision making. Technical skills are those that are not ANTS: basic and technical knowledge (gathering information, preparation, and working with protocols and checklists)11,19–23 and psychomotor skills (spatial perception, eye-hand coordination).8 The ANTS can also be divided into cognitive skills (decision making, planning, strategy, risk assessment, situation awareness) and interpersonal affective skills (teamwork, communication, leadership). Both are necessary for safe and effective performance24,25 and represent 2 legs in the skills triangle with psychomotor skills being the third leg (figure 1).7,8 The ANTS concept was designed using methods of task analysis similar to the model used for pilots.15,26 ANTS include the main nontechnical skills (cognitive and affective) associated with good anesthetic practice11,19,27 that should be specifically taught and evaluated in all anesthesiology training programs.28–30 Cognitive errors are thought process errors that lead to incorrect diagnoses or treatments. Understanding and correcting cognitive errors31 cannot be overemphasized (a table of ranked cognitive errors is provided as online supplemental material). A goal for each resident within an anesthesiology training program should be to explore, define, and pinpoint his or her own cognitive errors, and the program should plan an education strategy designed to decrease these errors. View larger version (22K) FIGURE 1Affective, Cognitive, and Psychomotor Skills Evaluated During Each Stage of a Scenario Affective (interacting) skills: (a) receiving information, (b) responding, (c) valuing, (d) organizing, and (e) commitment (see the affective circle); cognitive (thinking) skills: (a) basic knowledge, (b) comprehension, (c) application, (d) analysis, (e) evaluation, and (f) creation (see the cognitive circle); psychomotor (doing) skills: (a) technical information (spatial perception, guided response, eye-hand coordination), (b) no supervision, (c) technical complexity and