Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes

摘要

Bayesian optimization (BO) methods are useful for optimizing functions thatare expensive to evaluate, lack an analytical expression and whose evaluationscan be contaminated by noise. These methods rely on a probabilistic model ofthe objective function, typically a Gaussian process (GP), upon which anacquisition function is built. This function guides the optimization processand measures the expected utility of performing an evaluation of the objectiveat a new point. GPs assume continous input variables. When this is not thecase, such as when some of the input variables take integer values, one has tointroduce extra approximations. A common approach is to round the suggestedvariable value to the closest integer before doing the evaluation of theobjective. We show that this can lead to problems in the optimization processand describe a more principled approach to account for input variables that areinteger-valued. We illustrate in both synthetic and a real experiments theutility of our approach, which significantly improves the results of standardBO methods on problems involving integer-valued variables.