Application of the rule space model in computerized adaptive testing for diagnostic assessment.

摘要

With the development and maturation of computer technology and test theories, computerized adaptive testing (CAT) has become more widely used in educational assessment. Most CATS systems are based on item response theory (IRT), and as compared with traditional paper and pencil tests (P & P test), CAT selects the most suitable items according to individual examinee's ability. Thus, it provides a more accurate and efficient estimate of examinees' ability.;Along with the development of the society and the rising education for the general public, there is stronger demand that educational tests should not only be used to rank and screen examinees, but also to provide diagnostic information about students' misconceptions. Researchers have proposed different models to establish these diagnostic tests, through which we can estimate examinees' knowledge ability or psychological traits from their item responses. Among these models, the Rule Space Model (RSM) is an influential one and has been implemented in the analysis and reporting of the PSAT (Preliminary SAT Test) for high school leavers in the United States of America.;Incorporating RSM into CAT will lead to a totally new testing system---the diagnostic computerized adaptive test. However, we have little knowledge on the characteristics of the model in its ability estimation efficiency, its new item selection strategy, and stopping rule, among other things. The present research aimed to understand these characteristics.;Broadly differentiated, there were two main research purposes. Firstly, the research aimed to understand in greater depth the properties of the RSM, including, such as, how estimation accuracy would be affected by different factors (e.g. test length). Secondly, in the incorporation of RSM in diagnostic computerized adaptive tests, the performances of four different item selection methods were compared. These two purposes were achieved through two related sets of studies. The first set of experiments consisted of seven simulation experiments to investigate factors affecting the performance of the RSM. These factors being examined include: test length, number of attributes in the test, hierarchical relations among attributes, nature (simple versus complicated) of items used, item guessing or slipness parameter, and item response model (one- versus three-parameter model) used in item bank. The second set of experiments compared four item selection strategies in diagnostic CAT, with random selection method as borderline method. In order to calculate and compare the accuracy of examinees' attribute estimation, the Monte Carlo simulation method was used in both sets of studies. (Abstract shortened by UMI.)