Learning-based Cell Selection for Open-access Femtocell Networks

摘要

In an open-access femtocell networks, nearby cellular users (Macro User: MU) may join one of the neighboring femtocells to enhance their capacity through a handover procedure. To avoid undesirable effects such as the ping-pong effect after a handover, the effectiveness of cell selection method must be ensured. Previous work related to such a problem is based on instantaneous measure of single or multiple metrics, e.g. capacity, received signal strength (RSS), load, etc. However, one problem with such approaches is that present measured performance does not necessarily reflect the future performance, thus the need for novel cell selection that can predict the horizon. In this report, we propose a Reinforcement Learning (RL) Q-learning algorithm as a model-free solution for the cell selection problem in a non-stationary femtocell network. The MU takes advantage of the RL algorithm, during a handover decision, to estimate the efficiency of neighboring femtocells through trial-and-error interaction with its environment. The simulation results show the benefits of using learning in terms of the gained capacity and the number of handovers with respect to different selection methods in the literature (least loaded (LL), random and capacity-based).