Promoting robust black-box solvers through competitions

摘要

This paper presents the experiences of the organizers of the four constraint solver competitions which were held in conjunction with CP in the previous years. The paper mainly focuses on the competitions which were held in 2008 and 2009, outlines the reasons for organizing the competitions, describes how the solvers were evaluated, and presents lessons, observations, and general trends.