Research Program Peer Review: Purposes, Principles, Practices, Protocols

摘要

The purposes, principles, practices, and protocols of research program peer review are described. While the principles are fundamentally generic, and apply to peer review across the full spectrum of performing institutions as well as manuscript, proposal, and program peer review, the focus of this report is peer review of proposed and ongoing research programs in federal agencies. Following the self-contained Executive Summary of factors for high-quality peer reviews, the report addresses potential implications of the implementation of the Government Performance and Results Act of 1993 on federal agency research program peer review practices. Then, the report describes strengths and weaknesses of major peer review components and issues, including the following: Objectives and Purposes of Peer Review; Quality of Peer Review; Impact of Peer Review Manager on Quality; Selection of Peer Reviewers; Selection of Evaluation Criteria; Secrecy (Reviewer and Performer Anonymity); Objectivity, Bias, and Fairness of the Peer Review; Normalization of Peer Review Panels; Repeatability and Reliability of the Peer Review; Effectiveness and Predictability of Peer Review; Global Data Awareness; Costs of Performing a Peer Review; Ethical Issues in Peer Review; and Alternatives to Peer Review. The report then presents different federal agency peer review practices as well as sample protocols and processes for conducting a successful research program peer review. Some peer review variants, such as the Science Court and Network-Centric Peer Review, are described, and research requirements to improve peer review are discussed. The final section is an extensive bibliography of over 3,000 references that includes text references and related references for further reading. (49 refs.).