Cooperation action acquisition of the autonomous robot swarm by reinforcement learning in a seesaw balancing problem

摘要

This paper proposes a new approach to realize a reinforcement learning scheme for autonomous multiple agents system. In our approach, we treat the cooperative agents systems in which there are multiple autonomous mobile robots, and the seesaw balancing task is given. This problem is an example of corresponding tasks to find the appropriate locations for multiple mobile robots. And as another problem, in the environment where autonomous robots do cooperative action, there is a possibility that the whole system may have instability by an agent's failure. It is important practically to set up a control system in consideration of stability when an agent breaks down. Each robot agent on a seesaw keeps to be balanced state by using corresponding reinforcement learning systems. As a most useful algorithm of reinforcement learning, the Q-learning method is well known. However, feasible action values of robot agents must be categorized into some discrete action values. Therefore, in this study, the actor-critic method is applied to treat continuous values of agents actions. Each robot agent has a set of normal distribution, that determines a distance of the robot movement for a corresponding state of the seesaw system. Based on a result of movement in this system, the normal distribution is modified by actor-critic learning method. The simulation result shows the effectiveness of our approaching method.