Deconstructing a domain into its cognitive attributes: test construction and data analysis

摘要

If a cognitive model of learning provides the framework for both the educational system (i.e., curriculum, instruction, and assessment) and the design of the assessment, then both learning and instruction are optimized (Huff & Goodman, 2007). In this paper I describe the process of creating a cognitive framework for a college-level introductory statistics course. I begin with a review the current literature relating to cognitive diagnosis assessments and cognitive diagnosis models. Then I explain the procedure of deconstructing the course domain into a hierarchical arrangement of cognitive attributes and I explain the process of constructing a 50-item multiple-choice assessment that will be used to determine examinee mastery of the attributes. For the analysis of the examination data, I will present the item parameter estimates, examinee parameter estimates, and model fit statistics that were found using Item Response Theory, the Continuous Diagnostic Model, and the Fusion Model. Finally, I will discuss the compare examinee classification rates for simulated examination data that were found using a reduced version of the Option Based Fusion Model (which uses information from distractors or incorrect options) and the original Fusion Model. The results of the simulation study indicate that information is gained by using specific answer choices in the cognitive model.