A Role of the Asymptotic Equipartition Property in Return Maximization of Reinforcement Learning

摘要

Reinforcement learning is well-known as an effective framework to describe a decision-making process that consists of interactions between an agent and an environment. In the framework, an agent learns an optimal policy via return maximization, not via the instructed choices by a supervisor. The process treated in reinforcement learning is in general formulated as an ergodic Markov decision process and is designed by timing some parameters of the action-selection strategy so that the learning process eventually becomes almost stationary. In this paper, we examine a theoretical class of more general processes such that the agent can achieve return maximization by considering the asymptotic equipartition property of such processes. As a result, we show several necessary conditions that the agent and the environment have to satisfy for possible return maximization.