Comparing Methods for Item Analysis: The Impact of Different Item-Selection Statistics on Test Difficulty

摘要

Practitioners often depend on item analysis to select items for exam forms and have a variety of options available to them. These include the point-biserial correlation, the agreement statistic, the B index, and the phi coefficient. Although research has demonstrated that these statistics can be useful for item selection, no research as of yet has compared the difficulty of the items that these statistics will select in test development. The purpose of this research was to compare the difficulty of the items that these statistics would select for a test form and how the difficulty of item selection might vary, depending on the cut score. Results showed that considerable variability existed between the difficulty levels of the items selected by each of the statistics. The difference in difficulty depended on the location of the cut score. The agreement statistic tended to select the easiest items, whereas the point-biserial correlation selected items near the mean. Other results showed that the phi index tended to select items near the value of the cut score, whereas the B index followed this pattern to a lesser extent.