Boosting Reinforcement Learning in Competitive Influence Maximization with Transfer Learning

摘要

Companies aim to promote their products under competitions and try to gain more profit than other companies. This problem is formulated as a Competitive Influence Maximization (CIM). Recently, a reinforcement learning has been used to solve the CIM problem, that is, to find an optimal strategy against competitor in order to maximize the commutative reward under the competition from other agents. However, reinforcement learning agents require huge training time to find an optimal strategy whenever the settings of the agents or the networks change. To tackle this issue, we propose a transfer learning method in reinforcement learning to reduce the training time and utilize the knowledge gained on source network to target network. Our method relies on two ideas, the first one is the state representation of the source and target networks in order to efficiently utilize the knowledge gained on source network to target network. The second idea is to transfer the final Q-solution of source network while learning on the target network. We validate our transfer learning method in similar or different settings of source and target networks while competing against the competitor's known strategies. Experimental results show that our proposed transfer learning method achieves similar or better performance as a baseline model while significantly reducing training time in all settings.