An Adaptive Testing Strategy for Mastery Decisions

摘要

In an attempt to increase the efficiency of mastery testing while maintaining a high level of confidence for each mastery decision, the theory and technology of item characteristics curve (ICC) response theory and adaptive testing were applied to the problem of judging individuals' competencies against a prespecified mastery level to determine whether each individual is a 'master' or a 'nonmaster' of a specified content domain. Items from two conventionally administered classroom mastery tests administered in a military training environment were calibrated using the unidimensional three-parameter logistic ICC model. Then, using response data originally obtained from the conventional administration of the tests, a computerized adaptive mastery testing (AMT) strategy was applied in a real-data simulation. Results obtained from the AMT procedure were compared to results obtained from the traditional mastery testing paradigm in terms of the reduction in mean test length, information characteristics, and the correspondence between decisions made by the two procedures for three different mastery levels and for each of the two tests. The AMT procedure reduced the average test length 30% to 81% over all circumstances examined (with modal test length reductions of up to 92%), while reaching the same decision as the conventional procedure for 96% of the trainees. Additional advantages and possible applications of AMT procedures in certain classroom situations are noted and discussed.