The Formation of Homogeneous Item Sets When Guessing is a Factor in Item Responses

摘要

One of the major assumptions of latent trait theory is that the items in a test measure a single dimension. This report describes an investigation of procedures for forming a set of items that meet this assumption. Factor analysis, nonmetric multidimensional scaling, cluster analysis and latent trait analysis were applied to simulated and real test data to determine which technique could best form an unidimensional set of items. Theoretical and empirical evaluations were also made of the effects of guessing on the dimensionality of test data. The results indicated that guessing affected highly discriminating items more so than poorly discriminating items. However, the effect of guessing on the dimensionality of tests with common distributions of difficulty and discrimination indices was found to be minimal. Of the procedures evaluated for sorting items into unidimensional item sets, principal factor analysis of phi coefficients gave the best results overall. Nonmetric multidimensional scaling also showed promise when used with Yule's Y, phi, or tetrachoric similarity coefficients, but it did not perform as well as the factor analytic techniques on the real test data. In summary, guessing does have an effect on test data, but the effect is not very large unless items of extreme difficulty are present in the test. Of the procedures evaluated, traditional factor analytic techniques gave the most useful information for sorting test items into homogeneous sets. (Author)