Using Reward/Utility Based Impact Scores in Partitioning

摘要

Reinforcement learning with reward shaping is a well-established but often computationally expensive approach to multiagent problems. Agent partitioning can assist in this computational complexity by treating each partition of agents as an independent problem. We introduce a novel agent partitioning approach called Reward/Utility-Based Impact (RUBI). RUBI finds an effective partitioning of agents while requiring no prior domain knowledge, provides better performance by discovering a non-trivial agent partitioning, and leads to faster simulations. We test RUBI in the Air Traffic Flow Management Problem, where there are simultaneously tens of thousands of aircraft affecting the system and no intuitive similarity metric between agents. When partitioning with RUBI in the ATFMP, there is a 37% increase in performance, with a 510x speed up per simulation step over non-partitioning approaches.