An Adaptive Decision Making Approach Based on Reinforcement Learning for Self-Managed Cloud Applications

摘要

Cloud applications usually face challenges from the dynamic changing environment, growing number of users and large amount of concurrency accesses etc. A self-management based on resource adjustment is emerging to deal with these problems. To achieve self-management, it is the key problem to make resource adjustment decisions adaptively. There are two types of decision making in self-management: static and dynamic. Static method is not suitable for cloud environments. And the existing dynamic methods (i.e. intelligence optimization algorithm-based method and off-line model-based method) are often inefficient or not adaptive to the variation of environment. In this paper, we propose a reinforcement learning-based approach to dynamic decision making in resource adjustment-based self-management. This approach enables a cloud application to guarantee its performance by learning the results of its behavior and by dynamically changing its plans based on the learning in the presence of environmental changes. The process of our decision making approach for the self-managed cloud applications is also presented in this paper. Experimental results using a prototype framework in the context of a SBS application demonstrate the effectiveness of this approach.