Clique-based Cooperative Multiagent Reinforcement Learning Using Factor Graphs

摘要

In this paper,we propose a clique-based sparse rein-forcement learning(RL)algorithm for solving cooperative tasks. The aim is to accelerate the learning speed of the original sparse RL algorithm and to make it applicable for tasks decomposed in a more general manner.First,a transition function is estimated and used to update the Q-value function, which greatly reduces the learning time.Second,it is more reasonable to divide agents into cliques,each of which is only responsible for a specific subtask.In this way,the global Q-value function is decomposed into the sum of several simpler local Q-value functions.Such decomposition is expressed by a factor graph and exploited by the general max-plus algorithm to obtain the greedy joint action. Experimental results show that the proposed approach outperforms others with better performance.