Mini-Sessions on Assessment for Continuous Faculty Development

摘要

The Challenge As the Next Accreditation System (NAS) is rolled out, training programs will need to implement a process for continuous faculty development on the use of assessment tools. Reliable assessments of performance by multiple trained faculty are essential to demonstrate resident skill development over time. Yet, with increasing time demands on faculty, full- or half-day training sessions recommended by experts for greater efficacy of training are generally not feasible.1 Alternately, brief, continuous training, embedded in regularly scheduled meetings, can support ongoing faculty development to help meet NAS requirements.2;What Is Known Assessment tools are only as good as the individuals who use them.3,4 A heightened awareness among faculty about the purpose, use, and interpretation of assessment tools will develop shared mental models and more reliable performance assessments over time.5 Combining the results of 6 to 10 observations with other assessment instruments yields a “composite reliability” on which progress in professional development and performance can be documented in a reliable manner.6 Thus, it is imperative that faculty receive regular training on each assessment instrument to understand (1) the purpose and use of the tool (Behavior Observation Training [BOT]), (2) the definitions and criteria for each performance dimension on the tool (Performance Dimension Training [PDT]), and (3) the need for rating accurately and consistently with the tool (Frame of Reference Training [FORT]).3 To report the Milestone evaluations, Clinical Competency Committees (CCCs) will rely on composite scores from faculty assessments of residents. Given the importance of each faculty assessment, medical educators have the professional responsibility to become trained raters on their program assessment tools.3;A Guide to Developing Mini-Sessions on Assessment for Faculty Conduct a needs assessment with your CCC. Ask CCC members to identify assessment instruments and concomitant education needed to train faculty. For example, there might be concerns regarding the Milestone implementation, recent revisions to an assessment tool, or wide variation among faculty members' ratings resulting in inconsistent scoring. With the CCC, develop and prioritize a list of faculty training needs as the core content for the mini-session curriculum. Consider providing the CCC with a discussion resource, such as the Holmboe3(p119–124) chapter on “Direct Observation by Faculty.” Develop the mini-session curriculum. The overarching goal is for faculty to use program assessment tools with greater ease, higher frequency, and increased reliability. The number and order of sessions will depend on the identified list of assessment needs and will vary by program. Individual mini-sessions, based on the priority list of faculty training assessment needs above, might also include 1 or more of the following objectives: Discuss the purpose, uses, and context of the assessment tool (BOT). Review and discuss the definitions and criteria used in the assessment tool (PDT). Elicit examples of resident behaviors that exemplify specific ratings to achieve greater interrater reliability (FORT). Frame and deliver the mini-sessions. To support a continuous improvement cycle for faculty development, design a brief evaluation form for participants after each session. The form might include items on session relevancy, effectiveness, and usefulness (5-point Likert scale). Administer the form after every mini-session (t a b l e). View larger version (32K);How You Can Start TODAY Talk to your CCC for initial confirmation of faculty development needs and initiate preliminary planning. With the CCC, prepare a brief proposal, including rationale, needs, goals, curriculum, and framework. Secure a commitment from your chair/leader for 20?minutes during standing faculty meetings and retreats. Designate interested faculty, including members of your CCC, to lead sessions. Establish a