The Search for the Holy Grail: Content-Referenced Score Interpretations from Large-Scale Tests

摘要

The measurement industry is in crisis. The public outcry against "over testing" and the opt-out movement are symptoms of a larger sociopolitical battle being fought over Common Core, teacher evaluation, federal intrusion, and a host of other issues, but much of the vitriol is directed at the tests and the testing industry. If we, as measurement professionals, think that these critics just don't understand the complexities and challenges of measuring the new expectations for deeper learning, we could very well end up being seen as even more scientifically aloof than we already are. What's worse, we could end up being irrelevant to the larger policy conversations. I argue that at least part of the reason driving the over testing/opt-out movement is that too many stakeholders see too little value in the results of our large-scale assessments. How could that be? We put so much effort into carefully designing our assessments to make sure they meet rigorous psychometric criteria to accurately measure the target constructs. So why isn't the public more appreciative? Clearly the accountability uses (abuses) are conflated with the perceptions of the tests themselves, but if users were able to get a clear picture of what students actually know and are able to do, I think they would be less likely to want to opt out of receiving such valuable information. Briggs and Peck (this issue) are putting forth an approach for trying to extract more understandable and meaningful information from large-scale-test scores. They focus specifically on doing so when tests are vertically scaled across grades, but it is important to consider the more general application of their ideas to within-grade test score scales as well.