Integrating advice with reinforcement learning.

摘要

Reinforcement learning has proven to be an effective method for creating intelligent agents in a wide range of applications. However, it suffers from the need for a large number of training episodes, a problem that is especially noticeable in tasks that need to be learned on-line. Reinforcement learning further deteriorates in tasks that generate insufficient feedback and/or have long inter-reinforcement times.; We extend the traditional reinforcement learning approach with an external teacher that provides additional instruction to the learning agent. The agent incorporates both the reinforcements and the external instruction to obtain a combined policy that is correct with respect to the task and benefits from the teacher's advice. The learning agent converts the instructions to an extended or user reward function that, together with the task reward function, defines a composite reward function that more accurately defines the teacher's perception of the task.