Direct Policy Search Reinforcement Learning Based on Variational Bayesian Inference

摘要

Direct policy search is a promising reinforcement learning framework in particular for controlling continuous, high-dimensional systems. As one of direct policy search, reward weighted regression (RWR) was proposed by Peters et al. The RWR algorithm estimates the policy parameter based on EM algorithm and is therefore prone to overfitting. In this paper, we focus on variational Bayesian inference to avoid overfitting problem, and propose the direct policy search reinforcement learning based on variational Bayesian inference (VBRL). The performance of the proposed VBRL is assessed in several experiments with mountain car and ball batting task. These experiments highlight the VBRL produces higher average return and outperforms the RWR.