Environment-Adaptive Sizing and Placement of NFV Service Chains with Accelerated Reinforcement Learning

摘要

As network function virtualization spread, network service providers have been able to deliver various networks flexibly and rapidly. In particular, products and services that build network functions on a wide area network of organizations, such as enterprises, have been spreading. Since the user substrate environment and performance requirement differ in such services, optimal virtualized network function (VNF) resource sizing and placement need to be considered individually. To adapt to such environmental diversity, methods for applying reinforcement learning (RL), which includes an adaptive optimization mechanism, have been proposed. However, current RL methods have difficulty to complete learning on a real network because of too many required explorations. We propose an accelerated RL method that can learn proper VNF sizing and placement on a real network under various environments. Our method divides the RL process into two steps depending on the learning objective. We compared the proposed and a conventional RL methods through three scenarios with different substrates. We confirmed that the conventional RL method cannot learn properly even if it takes ten thousand explorations, whereas, our method can learn a cost-efficient resource sizing and placement that meets the performance requirements within only one thousand explorations.