A guide to statistical analysis for single criteria heuristics evaluation

摘要

The success obtained by metaheuristic techniques in resolving combinatory problems of a real character has led to a veritable explosion in the number of studies in this field. From a scientific perspective, these new approaches must demonstrate improved performance, normally with respect to prior proposals. This involves the carrying out of a series of computational experiments and statistical tests demonstrating their validity. However, in a large number of cases and for diverse causes (a lack of data relative to the benchmark heuristic or due to not knowing the most appropriate statistical tests), some authors fail to present conclusive statistical tests that may be qualified as possessing a minimum level of rigour according to the scientific method. This paper analyses the different tests that may be considered to verify significant differences between the performance of different single criteria heuristics, as well as including a number of application examples, and a discussion about the problem of robustness.