Objective: The purpose of this study was to evaluate student achievement and attitudes pertaining to a performance-based assessment in a medical student chest radiology curriculum using mixed quantitative and qualitative methods. Methods: One-hundred-and-sixty-one participating second-year students, in the 2015-16 academic year took a post-curriculum multiple choice question (MCQ) exam. Students also underwent a performance-based assessment in the form of a chest x-ray interpretation small-group session administered by radiology faculty at a picture archiving and communications system (PACS) workstation. Each student verbally interpreted one chest radiograph showing one of 6 pathologies and was given a numerical rating based on a standardized rubric. This score was compared to the correctness of the student?s answer to the corresponding MCQ question on the same topic. All students completed a post-session questionnaire. Open-ended free-text responses regarding student attitudes were coded into qualitative themes by three independent raters. High inter-rater agreement was demonstrated by average agreement index of .82 or greater (ranging from 0-1, with 1 indicating perfect agreement) for responses to the most frequent themes. Results: There was no significant association between scores on MCQ exam and performance-based assessment. Up to 90% of post-session questionnaire respondents indicated a Likert rating of 5 (strongly agree) when asked if the sessions improved understanding of chest x-rays and their ability to identify specific radiologic pathology. Dominant themes from open-ended responses were then derived. Conclusions: Lack of agreement between student performance on the chest x-ray interpretation small-group session and that on the parallel MCQ exam suggests that they each measure a different type of achievement, with the former emphasizing skills over knowledge. Features most commonly valued by students in the performance-based assessment were its ability to reinforce prior knowledge, supply an authentic and relevant PACS experience simulating real life, and provide an opportunity for active practice of radiology interpretive skills.
展开▼