An Empirical Analysis of Assessment Errors for Weights and Andness in LSP Criteria

摘要

We investigate the accuracy of parameters in the Logic Scoring of Preference (LSP) criterion functions for system evaluation. Main parameters are weights and conjunction/disjunction degrees (andness/orness). Weights reflect the level of relative importance of various decision variables. Andness/orness describes a desired level of simultaneity/replaceability in satisfying component criteria. These parameters are assessed by one or more professional evaluators and their values differ from (usually unknown) optimum values. In this paper we identify all potential sources of errors in LSP criterion functions. Our goal is to investigate the distribution of errors, their average values, and the quality of individual evaluators and their teams.