Peer Review of Search Strategies. Methods Research Report.

摘要

Many steps in the preparation of effectiveness and comparative effectiveness reviews (CERs) by the Agency for Healthcare Research and Quality's Effective Health Care (EHC) Program involve outside review and input (e.g., by Key Informants, the Technical Expert Panel, Draft Report Peer Reviewers). However, development of bibliographic database search strategies is currently not consistently peer reviewed. An opportunity exists for the EHC Program to leverage its expert searchers across the 14 Evidence-based Practice Centers (EPCs) in the Program by implementing a process of peer review of search strategies. Due to the number and frequency of observed errors in published search strategies uncovered in the research literature, an evaluation of the feasibility of instituting a peer review process is warranted. Specific Aims: 1. Evaluate whether the PRESS instrument or no-instrument ('free-form' evaluations) is preferred by Technical Expert Peer Reviewers (TEPRs) of search strategies. 2. Evaluate the usefulness of a peer review process for database search strategies, that is, do peer reviews change search strategies. 3. Evaluate the costs of implementing a formal peer review of search strategies program as a part of the review process. We identified current research protocol phase CER search strategies to review, TEPRs from across the EHC Program, and the original expert searchers who worked on developing each of the search strategies. Each TEPR was assigned to either the control group who only wrote a 'free-form' reviews or to the group who initially wrote a 'free-form' review, then trained using the PRESS Instrument, and finally completed the last review using the PRESS instrument. Original expert searchers were asked to comment on the reviews of their searches. One study researcher administered the peer review process and log time required to perform it, while the other, blinded study researcher analyzed qualitative and quantitative data derived from the reviews, as well as demographic information about the TEPR and original expert searchers.