How to compare many-objective algorithms under different settings of population and archive sizes

摘要

In the evolutionary multi-objective optimization community, algorithm comparison is usually performed under the same population size. However, this is not always fair because its best specification is usually different in each algorithm. In many-objective optimization, the number of solutions to be found may depend on the situation. If the decision maker wants to analyze the entire Pareto front, thousands of solutions may be needed. If the decision maker wants to choose a single final solution from some candidates after their quick checks, only a small number of representative solutions may be needed. In this paper, we discuss how to evaluate the ability of evolutionary many-objective optimization algorithms to find an arbitrarily specified number of non-dominated solutions. Our idea is the use of solution selection after the termination of each algorithm. We examine two scenarios: One is solution selection from the final population, and the other is from all of the examined solutions. Through computational experiments, first we demonstrate that performance comparison heavily depends on the population size. Then we examine the effects of solution selection from the final population and the examined solutions on comparison results.