A comparative study of Roth-Erev and modified Roth-Erev reinforcement learning algorithms for uniform-price double auctions.

摘要

This work focuses on multi-agent learning in market contexts. It reports findings from a comparative study of three reinforcement learning algorithms currently in use for a variety of market applications. Two double-auction market testbeds are developed and used to carry out benchmark comparisons involving intensive parameter sweeps with heat map visualization of parameter sensitivities. A primary concern is the degree to which each tested algorithm permits learning agents to converge to the choice of a best action measured in terms of accumulated profits. Some findings from a mathematical analysis of the algorithms' properties are also reported.; The three reinforcement learning algorithms studied in this work are: the Roth-Erev algorithm proposed by Erev and Roth [1], the Modified Roth-Erev (MRE) reinforcement learning algorithm proposed by Nicolaisen et al. [2] and the Variant Roth-Erev (VRE) learning algorithm proposed by Sun and Tesfatsion [3].