A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool

摘要

Background? The importance of effective clinical teaching skills is well established in the literature. However, reliable tools with validity evidence that are able to measure the development of these skills and can effectively be used by nonphysician raters do not exist.;Objective? Our initiative had 2 aims: (1) to develop a teaching development assessment tool (TDAT) that allows skill assessment along a continuum, and (2) to determine if trained nonphysicians can assess clinical teachers with this tool.;Methods? We describe the development of the TDAT, including identification of 6 global teaching domains and observable teaching behaviors along a 3-level continuum (novice/beginner, competent/proficient, expert) and an iterative revision process involving local and national content experts. The TDAT was studied with attending physicians during inpatient rounds with trained physician and nonphysician observers over 6 months.;Results? The TDAT showed emerging evidence of content, construct, and viable validity (the degree to which an assessment tool is practical, affordable, suitable, evaluable, and helpful in the real world) for the evaluation of attending physicians on inpatient rounds. Moderate to near perfect interrater reliability was seen between physician and nonphysician raters for the domains of promotion of clinical reasoning, control of the learning environment, ability to teach to multiple levels of learners, and provision of feedback.;Conclusions? The TDAT holds potential as a valid and reliable assessment tool for clinical teachers to track the development of each individual's teaching skills along the continuum from early development to mastery.;What was known and gap Teaching skills are important, yet there are no reliable tools for assessing them that can be used by nonphysician raters.;What is new A teaching development assessment tool showed acceptable validity and high interrater reliability.;Limitations Single institution, small sample study limits generalizability; validity evidence does not include comparisons with accepted measures of teaching proficiency.;Bottom line The tool can be used by trained nonphysician observers, and shows promise for tracking the development of teaching skills.;Editor's Note: The online version of this article contains the Cincinnati Children's Hospital General Pediatrics Master Educator Fellowship Teaching Development Assessment Tool–INPATIENT.;Introduction The development of an individual's skills as a clinical teacher occurs over time and, similar to other skills in medicine, can be tracked through the achievement of discrete observable milestones. While there are well-accepted skills for effective clinical teachers,1,2 previous studies of tools to assess these skills have not measured the longitudinal development of these skills.3–6 Most of these tools have been studied in simulated teaching sessions that utilize physicians or physicians-in-training as raters.7–11 Only a few were studied during live teaching encounters,3,5,7 and none to date have used nonphysician raters, a valuable quality in an era of increasing demands on physicians' time. Additionally, many of these tools do not have established evidence of validity. In 2011, a General Pediatric Master Educator Fellowship was established at Cincinnati Children's Hospital Medical Center (CCHMC) to formally train pediatricians in medical education.12 Our fellows, who are graduates of accredited pediatrics residencies, receive training in curriculum development, evaluation methodologies, and educational scholarship. They are also encouraged to develop their teaching skills in a variety of clinical settings for which we sought an assessment tool to document this development over time. However, we were unable to find an assessment tool with evidence of validity that tracked skill development longitudinally. To assess the evolution of one's teaching skills on inpatient rounds we sought to construct a novel teaching developme