Improving Reinforcement Learning Rates Using Prior Knowledge

摘要

This paper presents a new technique called Prior Policy Fallback (PPF) which provides a simple means of incorporating prior knowledge into a reinforcement learning system based on Q-learning. PPF performs significantly better than an intuitively similar method based on selectively initializing values of the Q table. The benefits of PPF occur because PPF does not adversely interact with the normal Q-learning update mechanisms. PPF simply accelerates the time to reward during initial trials and then gradually becomes less involved as normal Q-learning updates occur. PPF provides an alternative to manual teaching methods for accelerating learning during early trials and can be used in conjunction with many of the existing methods for accelerating Q-learning updates. The benefits of PPF are illustrated through several experiments based on a static grid-based world.