Stratified computerized adaptive testing: Further control on item exposure and extension to constrained situations.

摘要

In the last two decades, computerized adaptive testing (CAT) has become increasingly popular because of its many practical advantages and theoretical desirable characteristics.;Currently, most of the prevalent item selection algorithms are based on the traditional wisdom of the maximum information approach (MI), that is, the most informative item is chosen at each step of testing. The possible efficiency of the information approach, however, is likely to be at the expense of item security and the cost effectiveness in item pool management. The stratification approach that advocates a new philosophy of using less discriminating items first has been proposed recently by various researchers in an attempt to overcome these problems associated with the information driven selection algorithms. To date, several modifications of the stratified CAT designs have been developed to deal with different types of pool structure. Through simulated studies, these modifications have demonstrated the potential advantages of the stratification approach in having a better control on item security and a more balanced usage of items without sacrificing measurement efficiency. Previously, these stratified designs, namely Multistage a-Stratified Design (ASTR), a-Stratified Design with b-Blocking (BASTR), and Multiple Stratification Design (CBASTR), have been examined in unconstrained CAT environment. Their potential advantages found in such researches, therefore, cannot be automatically generalized to constrained situations. Furthermore, as their applicability to practically constrained settings is still much unknown, technical problems may exist when this new approach is being applied to CAT with practical constraints.;The research to be reported here consisted of a series of five simulation studies that addressed three main areas of investigation. First, several refinements of the stratification approach to meet stringent exposure control, content balancing, and other practical constraints had been identified and investigated. Second, a mixed design that captured the strengths of both the stratification and information approaches was developed and examined. And third, the advantages and disadvantages in terms of estimation accuracy and precision, item security control, pool utilization, and content validity of several enhanced stratified methods in handling stringent exposure control, content balancing, and multiple constraints were compared and studied in great depth. (Abstract shortened by UMI.)