New Feature Selection Method for Reinforcement Learning: Conditional Mutual Information Reveals Implicit State-Reward Dependency

摘要

Model-free reinforcement learning (RL) is a machine learning approach to decision making in unknown environment. However, real-world RL tasks often involve high-dimensional state space, and then standard RL methods do not perform well. In this paper, we propose a new feature selection framework for coping with high dimensionality. Our proposed framework adopts conditional mutual information between state and return sequences as a feature selection criterion, allowing the evaluation of implicit state-reward dependency. The conditional mutual information is approximated by a least-squares method, which results in a computationally efficient feature selection procedure. The usefulness of the proposed method is demonstrated on simulated mobile-robot navigation experiments.