Learning to balance a pole on a movable cart through RL: what can be gained using Adaptive NN?

摘要

The work of Barto, Sutton and Williams on the ACE/ASE model for Reinforcement Learning is here put in perspective. In their work a state-control (input-output) map, which allows to balance a pole hinged on a moving cart as long as possible, is learned when the only information provided by the environment is the system failure. This work has given rise to a large body of research in the fields of machine learning and artificial intelligence. Its relevance lies in the fact that it can be applied to control all those systems which are only partially known. A critical issue is the exploration of the state space which may require impractical amount of memory and learning time. Adaptive networks, which have been studied in the most recent years, offer a natural solution in the implementation of the learning system allowing an adaptive partitioning of the state space according to the task difficulty experienced in the different regions.