Adapting Accountability Systems to the Limitations of Educational Measurement

摘要

I am in more or less complete agreement with Professor Koretz's (this issue) characterization of the problem outlined in his paper, that current testing practices are not adequate for test-based accountability (TBA) systems, but I am far from sanguine about the proposed solution strategies. It is not surprising that the use of test scores for accountability would distort the meaning of the scores. As Donald Campbell suggested 40 years ago, in what has come to be known as Campbell's Law: The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." (Campbell, 1976, p. 49) In his valedictory, Cronbach (1995) reminded us that "when stakes for schools become significant, gamesmanship begins to take over" (p. 7) and that "if a hospital is told that it must gradually bring its mortality rate in surgery down to zero, the obvious ploy is to stop operating on patients who are at risk" (p. 8). Cronbach (1995) repeated a tale that he considered "too cute to be true," but made his point, and it is worth repeating. In a state where school sanctions were applied in the fourth grade, one school came up with the clever idea of not promoting into the fourth grade the weakest fraction of the third-grade class. They would be taught in a third-grade classroom for another year and, when the next assessment was safely over, would be boosted directly into Grade 5. Even if that story isn't true, there will be true stories like it. (Cronbach, 1995, p. 8).