Monte Carlo Study of Item Bias Detection in Multidimensional Tests

摘要

Persistent differences between racial groups on standardized aptitude test scores have suggested the potential for unfair discrimination against members of different racial and ethnic subpopulations. Because many occupational and educational opportunities are affected by mental test scores, the issue of test bias has consequences for many people in our society. Of the many statistical techniques proposed for detecting biased items there appears to be a preference for techniques based on a latent trait or item response theory (IRT) because sample estimates of population item parameters are invariant. This advantage occurs because, when the IRT model is valid, item parameters are invariant with respect to subpopulation ability distributions. This study concerns the effects of test multidimensionality on recommended item bias statistics. Simulation data samples (N=1,000 each) on a 50 item test were generated using a factor model described and used by Drasgow and Parsons. Subpopulation differences on common factors led to item bias that was identified to some extent by both chi-square and IRT bias indices. The signed indices were especially effective in distinguishing biased items from unbiased items. However, the use of either the signed chi-square or signed IRT index in multidimensional data clearly requires a priori knowledge of which subpopulation is at a disadvantage. This unexpected finding suggests further study of the properties of signed indices as well as a reevaluation of previous simulation research that has appeared to support their validity.