Comparison of Item Calibration Media in Computerized Adaptive Testing

摘要

A concern in computerized adaptive testing is whether data for calibrating itemscan be collected from either a paper-and-pencil (PP) or a computer administration of the items. Fixed blocks of power test items were administered by computer to one group of examinees and by pP to a second group. These data were used to obtain-based and PP-based three-parameter logistic model parameters of the items. Then each set of parameters was used to estimate item response theory pseudo-adaptive scores for a third group of examinees who had received all of the items by computer. The effect of medium of administration of the calibration items was assessed by comparative analyses of the adaptive scores using structural modeling. The results support the use of item parameters calibrated from either PP or computer administrations for use in computerized adaptive power tests. The calibration medium did not appear to alter the constructs measured by the adaptive test or the reliability of the adaptive test scores.