AdaPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems

摘要

Deep reinforcement learning (RL) has achieved remarkable advances in sequential decision making in recent years, often outperforming humans on tasks such as Atari games. However, model-free variants of deep RL are not directly applicable to physical systems because they exhibit poor sample complexity, often requiring millions of training examples on an accurate model of the environment. One approach to using model-free RL methods on robotic systems is thus to train in a relatively accurate simulator (a source domain), and transfer the policy to the physical robot (a target domain). This naive transfer may, in practice, perform arbitrarily badly and so online fine-tuning may be performed. During this fine-tuning, the robot may behave unsafely however, and so it is desirable for a system to be able to train in a simulator with slight model inaccuracies but still be able to perform well on the target system on the first iteration. We refer to this as the zero-shot policy transfer problem.