Equating Angoff Standard-Setting Ratings with the Rasch Model

摘要

A key part of determining cut-scores when performing Angoff standard setting is utilizing equating methods to place standard-setting ratings onto the scale used to report scores to examinees. This article describes three equating methods that can be employed to place Angoff ratings onto the scale used to report scores to examinees when applying the Rasch model. It is shown that the three methods make different assumptions and that the commonly observed situation of raters regressing Angoff standard-setting ratings toward the middle of the probability scale can impact the methods in different ways. Results from a simulation study suggest that equating standard-setting ratings at the individual panelist level may better reduce the impact of raters regressing their ratings toward the middle of the probability scale for hard and easy items. Analyses of data from two standard settings suggest that there can be important differences in cut-scores when the three methods are applied in practice. The study concludes with practical considerations related to using the three equating methods to determine cut-scores.