A method of automatic grade calibration in peer assessment

摘要

Once the exclusive preserve of small graduate courses, peer assessment is being rediscovered as an effective and efficient learning tool in large undergraduate classes, a transition made possible through the use of electronic assignment submissions and web-based support software.Asking large numbers of undergraduates to grade each others work raises a number of obvious concerns. How will mark reliability and validity be maintained? Can plagiarism be detected or prevented? What effect will "rogue" reviewers have on the integrity of the process? Will effective learning actually occur?In this paper we address the issue of grade reliability, and present a novel technique for identifying and minimising the impact of "rogues." Simulations suggest the method is effective under a wide range of conditions.