How assignment evaluations of writing students may inform the peer review process for scientific articles

摘要

In this paper, we present an assignment evaluation method, which has the potential to improve two aspects of the current peer review process for scientific articles: accountability of reviewers and consistency in evaluations. The method was designed to evaluate assignment quality on a post graduate technical writing course in a UK University, and incorporates qualitative and quantitative aspects. It helps tutors to remain thorough, objective and systematic, and to derive numerical grades from professional opinions. It is also used to teach academic standards for writing and to make tutors accountable for their judgments in feedback to students, thus prompting dialog and a sharing of knowledge, which is valuable to all communities of practice. For the peer review process of scientific articles, therefore, the method has the potential not only to introduce accountability and consistency in evaluations, but also to benefit the academic community through open and explicit articulation of disciplinary and research knowledge. We describe the system, how it is currently used, and report preliminary evidence suggesting that the system supports consistency.