Attritable Multi-Agent Learning

摘要

Autonomous systems will operate in highly contested environments in which it must be assumed that adversaries are equally capable, agile and informed. To achieve and sustain dominant performance in such environments, autonomous systems must be able to adapt through online machine learning while managing and tolerating attrition - that is, improve their performance quickly, even over the duration of a single engagement with principled asset losses. However, there are novel challenges to adapting effectively in such environments. We present an approach that leverages several recent innovations in reinforcement learning, distributed computing and trusted consensus algorithms such as Blockchain. We note that multi-agent systems operating in contested environments must leverage their redundancy for learning while also remaining resilient with respect to component failures and com- promises. In particular, to enable and accelerate learning, such systems will have to allow some number of components to operate sub-optimally to achieve the right exploration-exploitation balance needed for rapid and effective learning. At the same time that some number of components are possibly being sacrificed due to sub-optimal performance, the underlying mission of the system must be maintained. This leads to challenges in distributed trusted computing such as Byzantine agreement problems. Simulations demonstrating these various tradeoffs using epidemiological models are presented.