Distributed multi-agent temporal-difference learning with full neighbor information

摘要

This paper presents a novel distributed multi-agent temporal-difference learning framework for value function approximation,which alows agents using all the neighbor information instead of the information from only one neighbor.With full neighbor information,the proposed framework(1)has a faster convergence rate,and(2)is more robust compared to the state of-the art approaches.Then we propose a distributed multi-agent discounted temporal dfferene algorithm and a distributed muli-agent average cost temporal diference leaming algorithm based on th framework.Moreover,the two proposed algorthms'theoretical convergence proofs are provided.Numerical simulation resuts show that our proposed algorihms are superior to the gossip-based algorithm in convergence speed,robustness to noise and time-varying network topology.