Assessing scientific research activity evaluation models using multivariate analysis

摘要

The authors of this paper propose a method, based both on confirmatory and exploratory data analysis, aiming to assess the variability arising from the Composite Indicators (CIs) construction process. This research refers to an evaluation exercise very important for universities: the assessment of scientific research. The aim of every evaluation system is to synthesize all the information collected at universities into a unique CI, which will allow comparison of performances or ranks of the objects under evaluation. Since the methodology adopted to construct the CI is just one possible solution among several acceptable alternatives, it is reasonable to wonder about the results from the other options. The proposed approach investigates the impact of the different sources of variability occurring in CIs construction, also taking into account the external information available for each statistical unit. The term CI variability is used in the meaning of CI stability and it refers to differences emerging among CIs obtained using different subjective choices to construct the CI. Furthermore, the stability of the results is assessed through a combination of graphical tools and resampling methods. An empirical analysis is provided to discuss the methodological proposal. The research refers to the 'University Planning and Evaluation 2007-2009' system, implemented by the Italian government to finance public universities.