The Use of Information from Wrong Responses in Measuring Students' Achievement

摘要

Modern test theory is oriented toward measuring individual performances, regardless of the group or the population which the examinee comes from. In accordance with this trend, there is a noticeable shift from norm-referenced tests to criterion-referenced tests in the area of achievement testing. Latent trait models are becoming very popular due to their desirable properties which enable one to get item-free, population-free measures. As good teachers had already realized long ago, a lot of valuable information can be gained by analyzing the students' wrong responses. When a student answers a free response item (s)he gives the response which (s)he considers to be the correct one. Therefore, diagnosing the algorithm that led the student to his/her answer provides an important source of information for asessing his/her achievement. The purpose of this technical report is to provide the reader with some empirical results of error analysis in simple, signed number problems. Types of errors (or wrong algorithms) and their consistency will be defined and discussed. Since a wrong algorithm may sometimes lead to a correct answer the conventional scoring system will be compared with one based on error analysis (i.e., counting a response as correct only if the 'correct' algorithm was used) in terms of reliability, latent trait estimates, and the underlying dimensionality.