Construct Validity Evidence Based on Internal Structure: Exploring and Comparing the Use of Rasch Measurement Modeling and Factor Analysis with a Measure of Student Motivation

摘要

The current study examined and compared the use of Rasch measurement, common factor analysis (EFA), and confirmatory factor analysis (CFA) in establishing construct validity evidence based on internal structure with multi-item scales measuring middle and secondary studentsu27 achievement goal orientation and academic self-efficacy. 1054 complete responses were received to 18 items measuring mastery, performance-approach, performance-avoidance, and self-efficacy. Items in each subscale were first analyzed by each method as a unidimensional unit. Items were then analyzed by each method as a multidimensional unit. Results showed that when scales were analyzed individually, all three methods corroborated unidimensionality; however, when all items were analyzed together, the multi-factor model identified through EFA was not supported by the Rasch or CFA analysis. While EFA provided the best information about individual item functioning, Rasch provided important, additional information about rating scale functioning and item fit that helps diagnose poorly performing items. Results also support the use of EFA in evaluating the suitability of a scale for meeting the unidimensionality requirement of item response models. While the mastery and efficacy scales performed in a manner consistent with the motivation literature, the approach and avoidance subscales did not. Conclusions from the study include the need for better explication of conjoint use of classical and modern test theory methods in instrument development, more exposure of current and future researchers to the foundations of measurement theory, and more research about the saliency of measuring performance-avoidance.