Pass-Fail Decisions for Borderline Performers After a Summative Objective Structured Clinical Examination

摘要

>Objective. To determine what expert assessors value when making pass-fail decisions regarding pharmacy students based on summative data from objective structured clinical examinations (OSCE), and to determine the reliability of these judgments between multiple assessors.>Methods. All assessment data from 10 exit-from-degree OSCE stations for seven borderline pharmacy students (determined by standard setting methods) and one control was given to three of eight assessors for review. Assessors determined an overall pass-fail decision based on their perception of graduate competency. Assessors were interviewed to determine their decision-making rationale. Intraclass correlation coefficients were used to calculate reliability between assessor judgments.>Results. Expert consensus was achieved for three of the eight students, however, the assessors’ decisions did not align with standard-setting results. The reliability of assessors’ decisions was poor. Assessors focused on ability to make correct recommendations rather than on gathering information or providing follow-up advice. Global evaluations (including a student’s communication skills) rarely influenced the assessors’ decision-making.>Conclusion. When faced with making pass-fail decisions for borderline students, the assessors focus on evaluating the same competencies in the students but differed in their expected performance levels of these competencies. Pass-fail decisions are primarily based on task-focused components instead of global components (eg, communication skills), despite that global components are weighted the same for scoring purposes.