Scientometric indicators: peer-review, bibliometric methods and conflict of interests

摘要

The paper discusses the role of scientometric indicators in peer-review selection of research project proposals. An ex post facto evaluation was made of three calls for research project proposals in Slovenia: 2003 with a peer review system designed in a way that conflict of interest was not avoided effectively, 2005 with a sound international peer-review system with minimized conflict of interest influence but a limited number of reviewers, and 2008 with a combination of scientometric indicators and a sound international peer review with minimized conflict of interest influence. The hypothesis was that the three different peer review systems would have different correlations with the same set of scientometric indicators. In the last two decision-making systems (2005 and 2008) where conflict of interest was effectively avoided, we have a high percentage (65%) of projects that would have been selected in the call irrespective of the method (peer review or bibliometrics solely). In contrast, in the 2003 call there is a significantly smaller percentage (49%) of projects that would have been selected in the call irrespective of the method (peer review or bibliometrics solely). It was shown that while scientometric indicators can hardly replace the peer-review system as the ultimate decision-making and support system, they can reveal its weaknesses on one hand and on the other can verify peer-review scores and minimize conflict of interest if necessary.