Using the multivariate multilevel logistic regression model to detect DIF: A comparison with HGLM and logistic regression DIF detection methods

摘要

This study presents the Multivariate Multilevel Logistic Regression (MMLR) models to detect Differential Item Functioning (DIF), which are likely to detect DIF when the responses of an examinee are not locally independent. The study also compares the uses of the three MMLR models, three modified versions of Kamata's Hierarchical Generalized Linear Model (HGLM) and the standard logistic regression model as DIF detection methods. The comparison between these statistical procedures for DIF detection will be made using Michigan Educational Assessment Program reading test and simulated data. The simulation study evaluates their performances in the detection of uniform DIF. Simulated data are generated by the 3-parameter logistic Item Response Theory models, varying conditions of different sample size (400, 700, and 1000 examinees), test length (20, 40 and 60 items), the difference of parameter b (0.25, 0.50, and 0.75) and the ability distributions with different means and variances for the reference and focal groups. These test conditions are crossed completely and replicated 500 times. In these analyses, total score and IRT ability estimate are respectively used as the matching variable. The results show that MMLR can be used for DIF detection. It is also found that the heterogeneous variances of the two groups influence power and Type I error rates of these methods, and the HGLM DIF models are unsuitable to identify DIF.