Building the Plane As We Fly It

摘要

With the launch of the Outcomes Project in 2001, the graduate medical education (GME) community began the transition to competency-based medical education (CBME). While this launch promises revolutionary change to the status quo of GME, it is safe to say that the work of transitioning to a competency-based paradigm has been a slow evolutionary process that remains a work in progress. Given the enormous task of implementing true competency-based training, most educators realize that for the foreseeable future, GME will exist as a hybrid model of traditional and CBME components.1 The release of the Accreditation Council for Graduate Medical Education milestones, developed from the existing literature and expert consensus on dimensions of physician performance that are relevant in practice, hold significant promise to advance this field. However, the milestones are not a finished product. They must be vetted and refined, and faculty must be given specific professional development in the appropriate use of this framework. In this issue of the Journal of Graduate Medical Education, Dehon et al2 report on the use of a novel milestone-based, end-of-shift evaluation (ESE) of emergency medicine residents. This study highlights the challenge of advancing competency-based assessment. Using this ESE, the authors identified grade inflation or “milestone inflation,” and concluded that use of milestones as a stand-alone assessment tool is not advised. While their findings question the utility of this milestone-based assessment strategy, caution is advised when applying their findings to milestone-based assessment in general. To appreciate the challenge of developing milestone-based assessment, a review of a number of basic principles of CBME and of milestones is helpful. CBME is an outcomes-based approach to the design, implementation, and evaluation of a medical education program using an organizing framework of competencies.3 Rather than relying on time and process measures as proxies of competence, CBME aims to document the developmental progression of competence through authentic, preferably work-based assessment. This fundamental difference has profound implications regarding the learning environment, expectations of both faculty and learners, and the approach to assessment. Medical educators must appreciate these differences to most effectively develop and pilot milestones and competency-based innovations. Key concepts include: 1. Milestones, written as measureable and observable behaviors, define a developmental continuum of knowledge, skills, and attitudes in each of the general competencies. Essentially acting as a roadmap, milestones are intended to facilitate criteria-referenced and preferably work-based assessment of trainees. With appropriate faculty development, the developmental continuum they describe can create a common understanding of competence to inform the use of specific assessment tools, such as direct observation or chart-stimulated recall. Written in narrative form, milestones capitalize on literature identifying the value of narrative descriptions rather than numerical scales for assessment.4 The milestones describe the gradual, developmental acquisition of competence and are specifically intended to provide a framework informing formative assessment and feedback.5,6 Finally, the milestones are formulated to serve as a framework or guide for longitudinal assessment and a group decision process for determining competence, not a focal point in time assessment.7 Using specific milestones abstracted from that developmental continuum to inform yeso decisions in the very brief assessment window of a single emergency medicine shift risks loss of these benefits, and may potentially explain the disappointing performance of the ESE reported by Dehon et al.2 The use of milestones as a simple checklist also risks the deconstruction of the highly complex work of patient care. This realization, and the recognition that the res