Grounding Abstractions in Predictive State Representations

摘要

This paper proposes a systematic approach of representing abstract features in terms of low-level, subjective state representations. We demonstrate that a mapping between the agent's predictive state representation and abstract features can be derived automatically from high-level training data supplied by the designer. Our empirical evaluation demonstrates that an experience-oriented state representation built around a single-bit sensor can represent useful abstract features such as "back against a wall", "in a corner", or "in a room". As a result, the agent gains virtual sensors that could be used by its control policy.