Criterion-Related Validity of Adaptive Testing Strategies

摘要

Criterion-related validity of two adaptive tests was compared with a conventional test in two groups of college students. Students in Group 1 (N=101) were administered a stradaptive test and a peaked conventional test; students in Group 2 (N=131) were administered a Bayesian adaptive test and the same peaked conventional test. All tests were computer-administered multiple-choice vocabulary tests; items were selected from the same pool, but there was no overlap of items between the adaptive and conventional tests within each group. The stradaptive test item responses were scored using four different methods (two mean difficulty scores, a Bayesian score, and maximum likelihood) with two different sets of item parameter estimates, to study the effects on criterion-related validity of scoring methods and/or item parameter estimates. Criterion variables were high school and college grade-point averages (GPA), and scores on the American College Testing Program (ACT) achievement tests. Results indicated generally higher validities for the adaptive tests; at least one method of scoring the stradaptive tests resulted in higher correlations than the conventional test with seven of the eight criterion variables (and equal correlations for the eighth), even though the stradaptive test administered over 25% fewer items, on the average, than did the conventional test.