Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis

摘要

Objective The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance of a medication visit. This study examines two dimensions of validity for the P-SCO: internal structure and how scores correlate with another variable associated with competence (experience). Methods The faculty completed 601 P-SCOs over 4?years. Multilevel exploratory factor analysis was performed with minimum thresholds for eigenvalue (≥?1.0) and proportion of variance explained (≥?5.0%). Internal reliability was assessed with Cronbach alpha. To examine how scores changed with experience, mean ratings (1–4 scale) were calculated for each factor by quarter of the academic year. Separate linear mixed models were also performed. Results The analysis yielded three factors that explained 50% of the variance and demonstrated high internal reliability: affective tasks (alpha?=?0.90), cognitive tasks (alpha?=?0.84), and hard tasks (alpha?=?0.74). Items within “hard tasks” were assessment of substance use, violence risk, and adherence, and inquiry about interactions with other providers. Monitoring adverse effects did not load on the hard task factor but also had overall low mean ratings. Compared to the first quarter, fourth quarter scores for affective tasks ( b ?=?0.54, p ?