AgentP classifier system: self-adjusting vs. gradual approach

摘要

Learning classifier systems belong to the class of algorithms based on the principle of self-organization and evolution and have frequently been applied to mazes, an important type of reinforcement learning problem. Mazes may contain aliasing cells, i.e. squares in a different location that look identical to an agent with limited perceptive power. Mazes with aliasing squares present a particular difficult learning problem. As a possible approach to the problem, AgentP, a learning classifier system with associative perception, was recently introduced. AgentP is based on the psychological model of associative perception learning and operates explicitly imprinted images of the environment states. Two types of learning mode are described: the first, self-adjusting AgentP, is more flexible and adapts rapidly to changing information; the second, gradual AgentP, is more conservative in drawing conclusions and rigid when it comes to revising strategy. The performance of both systems is tested on existing and new aliasing environments. The results show that AgentP often outperforms (and always at least matches) the performance of other techniques and, on the large majority of mazes used, learns optimal or near optimal solutions with fewer trials and a smaller classifier population.