An incremental state-space construction based on the notion of contradiction for reinforcement learning

摘要

In this paper, we propose an incremental state-space construction method using ART neural network in order to construct appropriate state-space for reinforcement learning. The proposed method is inspired by the notion of contradiction studied by Piagget. In this method, a state-transition table which represents the learner's states and actions is recorded. Then, if the current state transition against a certain perception is in conflict with the record, a new state for such perception is generated. We introduce two kinds of contradiction: "a contradiction such that different results are caused by the same states and the same actions" and "a contradiction due to ambiguous states" Several computer simulations on pole-balancing problem and light seeking problem for autonomous mobile robots confirm us the effectiveness of the proposed state-space construction method.