Rough Sets-Based Prototype Optimization in Kanerva-Based Function Approximation

摘要

Problems involving multi-agent systems can be complex and involve huge state-action spaces, making such problems difficult to solve. Function approximation schemes such as Kanerva coding with dynamic, frequency-based prototype selection can improve performance. However, selecting the number of prototypes is difficult and the approach often still gives poor performance. In this paper, we solve a collection of hard instances of the predator-prey pursuit problem and argue that poor performance is caused by inappropriate selection of the prototypes for Kanerva coding, including the number and allocation of these prototypes. We use rough sets theory to reformulate the selection of prototypes and their implementation in Kanerva coding. We introduce the equivalence class structure to explain how prototype collisions occur, use a reduct of the set of prototypes to eliminate unnecessary prototypes, and generate new prototypes to split the equivalence classes causing prototype collisions. The Rough Sets-based approach increases the fraction of predator-prey test instances solved by up to 24.5% over frequency-based Kanerva coding. We conclude that prototype optimization based on rough set theory can adaptively explore the optimal number of prototypes and greatly improve a Kanerva-based reinforcement learner's ability to solve large-scale multi-agent problems.