Coordinating SON instances: Reinforcement learning with distributed value function

摘要

With the emergence of Self-Organizing Network (SON) functions network operators are faced with a practical problem: coordination of SON instances. The SON functions are usually designed in a standalone manner, i.e. they do not take into account the possibility that other instances of the same or different SON functions may be running in the network. This creates the risk of conflicts and network instability. Therefore a SON COordinator (SONCO) is needed. In this paper we design an operator centric SONCO that sees the SON instances as black-boxes, i.e. it does not know the algorithm inside the SON functions. Our aim is to improve the network stability (i.e. number of parameter changes) for SON instances of the same SON function. We employ Reinforcement Learning (RL) in order to profit from the information on the past SONCO decisions. We simplify the expression of the action-value function and we use state aggregation to further reduce the required state space, making it scale linearly with the number of coordinated cells. We provide a study case with the Mobility Load Balancing (MLB) function independently instantiated on every cell. The results show that the proposed SONCO improves the network stability.