Updating Strategies of Policies for Coordinating Agent Swarm in Dynamic Environments

摘要

This paper proposes strategies for updating action policies in dynamic environments, and discusses the influence of learning parameters in algorithms based on swarm behavior. It is shown that inappropriate choices for learning parameters may cause delays in the learning process, or lead the convergence to an unacceptable solution. Such problems are aggravated in dynamic environments, since the fit of algorithm parameter values that use rewards is not enough to guarantee a satisfactory convergence. In this context, strategy-updating policies are proposed to modify reward values, thereby improving coordination between agents operating within dynamic environments. A framework has been developed which iteratively demonstrates the influence of parameters and updating strategies. Experimental results are reported which show that it is possible to accelerate convergence to a consistent global policy, improving the results achieved by classical approaches using algorithms based on swarm behavior.