Investigation of a Nonparametric Procedure for Assessing Goodness-of-Fit in Item Response Theory

摘要

Tests of model misfit are often performed to validate the use of a particular model in item response theory. Douglas and Cohen (2001) introduced a general nonparametric approach for detecting misfit under the two-parameter logistic model. However, the statistical properties of their approach, and empirical comparisons to other methods, have not been examined. In the present study, a Monte Carlo simulation study was used to examine the empirical Type I error rates and power of two nonparametric statistics based on the Douglas and Cohen (2001) approach. The procedures are compared to two commonly used goodness-of-fit statistics, S-X2 (Orlando & Thissen, 2000) and BILOG's G2 (Mislevy & Bock, 1990), across conditions varied by test length, sample size, and the percentage of misfitting items. Overall, the nonparametrically based statistics controlled the Type I error rate and exhibited the most power across all conditions. Due to its close association with a graphical representation of the item response function, it is argued that the Douglas and Cohen (2001) approach may also allow for a more informative inspection of the type of misfit that is present in test items, although at a slightly greater computational cost.