Reinforcement learning with nonstationary reward depending on the episode

摘要

A model which represents nonstationary reward is proposed for reinforcement learning(RL). RL is a framework that the agent learns by the interaction with an environment. The agent receives the reward, and learns its behavior. The reward is determined by the designer. It is not necessary to design the behavior of the agent so that RL is expected to be applied to various applications. However, conventional RL algorithms work under the assumption that the environment is stationary. In other words, conventional RL can not accept unstationary rewards and the change of the objective. From the point of view of real world applications, it is necessary for the agent to deal with a change of the objective. In this paper, a learning technique to deal with a temporal change of the reward is proposed. In the proposed reward representation, the reward is divided into two parts: episode-dependent part and episode-independent part. The simulation experiments show the effectiveness of the proposed method.