Aligning individual and collective welfare in complex socio-technical systems by combining metaheuristics and reinforcement learning

摘要

In complex socio-technical systems it is not easy to find a balance between the welfare state (i.e., a state where the overall performance of a system is optimal) and a situation in which individual components act selfishly to optimize their own utilities. This is even harder when individuals compete for scarce resources. In order to deal with this, some forms of biasing the optimization process have been proposed. However, mostly, such approaches only work for cooperative scenarios. When resources are scarce, the components of the system compete for them, thus approaches designed for cooperative systems are not necessarily appropriate. In the present paper an approach is proposed, which is based on a synergy between: (i) a global optimization process in which the system authority employs metaheuristics, and (ii) reinforcement learning processes that run at each component or agent. Both the agents and the system authority exchange solutions that are incorporated by the other party. The contributions of the proposed approach are twofold: a general scheme for such synergy is given and its benefits are shown in scenarios related to selfish routing, a typical load balance problem in a complex socio-technical system.