A Proactive Q-Learning Approach for Autoscaling Heterogeneous Cloud Servers

摘要

Cloud providers offer different physical or virtual machine (VM) types that have different computational power and cost. Choosing the right configuration in a such heterogeneous environment able to sustain a workload while minimising costs is a challenging key aspect. Furthermore, turning-on/off a VM does not come for free, but introduce a reconfiguration overhead that might bring additional costs (e.g. time for moving to the new state and wasted resources for reconfiguration process). In this paper, we aim to find at run time a configuration s.t. (i) is able to sustain an input workload, (ii) does not over-provide resources, and that (iii) is as close as possible to the current one, to minimise the number of involved VMs in the reconfiguration, and thus, minimise the reconfiguration overhead. We propose here a Q-Learning approach to automatically learn the best policy to move from a configuration to another according to a predicted workload. We defined two reward functions which respectively look for (i) a configuration which perfectly fits the requested workload and (ii) a configuration which arrives close to the requested workload, to minimise the reconfiguration overhead. We compared the results with the two reward functions in term of average number of VMs involved in a reconfiguration and we show as with the first reward function we need to change in average 2.3 VM/reconfiguration while with the second reward function we can reduce such number up to 1 VM per reconfiguration with some over-provisioning.