Evaluation of One- and Three-Parameter Logistic Tailored Testing Procedures for Use with Small Item Pools

摘要

A two-stage study was conducted to compare the ability estimates yielded by tailored testing procedures based on the one-parameter logistics (1PL) and three-parameter logistics (3PL) models. The first stage of the study employed real data, while the second stage employed simulated data. In the first stage, response data for 3000 examinees were obtained for the 40 item ACT Assessment Mathematics Usage subtest. The first 2000 cases were used to obtain item parameter estimates for both models. Using these estimates, 1PL and 3PL tailored tests were simulated using the response data for the remaining 1000 cases. Both tailored testing procedures employed maximum likelihood ability estimation and maximum information item selection procedures. The two sets of ability estimates were then compared. In the second stage, response data for 3000 cases were simulated using the 3PL item parameter estimates from the first stage as true parameters. True abilities were selected from the standard normal distribution. The first 2000 cases were used for 1PL and 3PL calibration of the items, and the remaining 1000 cases were used to simulate 1PL and 3PL tailored tests. The two sets of ability estimates were compared to each other and to the true ability parameters. Results of both stages of the study indicated that the 1PL and 3PL tailored tests yielded highly correlated ability estimates, and there was no apparent advantage in terms of ability estimation to using one of the models over the other. Because the 1PL procedure was less expensive to use, it was the recommended model for this application. (Author)