Lenient Learning in Independent-Learner Stochastic Cooperative Games

摘要

We introduce the Lenient Multiagent Reinforcement Learning2 (LMRL2) algorithm for independent-learner stochasticcooperative games. LMRL2 is designed to overcome a pathologycalled relative overgeneralization, and to do so whilestill performing well in games with stochastic transitions,stochastic rewards, and miscoordination. We discuss the existingliterature, then compare LMRL2 against other algorithms drawnfrom the literature which can be used for games of this kind:traditional (a€?Distributeda€?) Q-learning, Hysteretic Q-learning,WoLF-PHC, SOoN, and (for repeated games only) FMQ. The resultsshow that LMRL2 is very effective in both of our measures(complete and correct policies), and is found in the top rankmore often than any other technique. LMRL2 is also easy to tune:though it has many available parameters, almost all of them stayat default settings. Generally the algorithm is optimally tunedwith a single parameter, if any. We then examine and discuss anumber of side-issues and options for LMRL2. color="gray">