Conditional Precision of Measurement for Test Scores: Are Conditional Standard Errors Sufficient?

摘要

This inquiry is focused on three indicators of the precision of measurement-conditional on fixed values of theta, the latent variable of item response theory (IRT). The indicators that are compared are (1) The traditional, conditional standard errors, sigma(eX|theta) = CSEM; (2) the IRT-based conditional standard errors, sigma irt(eX|theta)=CirtSEM=root 1/I(theta,X) (where I(theta,X) is the IRT score information function); and (3) a new conditional reliability coefficient, rho(X,X '|theta). These indicators of conditional precision are shown to be functionally related to one another. The IRT-based, conditional CSEM, CirtSEM, and the conditional reliability, rho(X,X '|theta), involve an estimate of the conditional true variance, sigma 2(tX|theta), which is shown to be approximately equal to the numerator of the score information function. It is argued-and illustrated with an example-that the traditional, conditional standard error, CSEM, is not sufficient for determining conditional score precision when used as the lone indicator of precision; hence, the portions of a score distribution, where scores are most-and-least precise, can be misidentified.