Adaptive Simulation-Based Training of Artificial-Intelligence Decision Makers Using Bayesian Optimization

摘要

This work studies how an artifical-intelligence-controlled dogfighting agent with tunable decision-making parameters can learn to optimize performance against an intelligent adversary, as measured by a stochastic objective function evaluated on simulated combat engagements. Gaussian process Bayesian optimization techniques are developed to automatically learn global Gaussian process surrogate models, which provide statistical performance predictions in both explored and unexplored areas of the parameter space. This allows a learning engine to sample full-combat simulations at parameter values that are most likely to optimize performance and provide highly informative data points for improving future predictions. However, standard Gaussian process Bayesian optimization methods do not provide a reliable surrogate model for the highly volatile objective functions found in aerial combat and thus do not reliably identify global maxima. These issues are addressed by novel repeat sampling and hybrid repeat/multipoint sampling techniques. Simulation studies show that hybrid repeat/multipoint sampling improves the accuracy of Gaussian process surrogate models, allowing artificial-intelligence decision makers to more accurately predict performance and efficiently tune parameters.