Optimised Q-learning for WiFi offloading in dense cellular networks

摘要

WiFi traffic offloading is becoming especially appealing because of the upcoming ultra-dense cellular networks. However, WiFi offloading decision as well as WiFi-Access Points (W-AP) selection should be carefully studied in order not to affect the offloaded users' experience. Here, a new reinforcement-learning framework is presented. The authors propose a distributed Q-learning algorithm in which each cellular user learns about his local environment and selects the best base station (macro-BS or W-AP) after reaching convergence. They introduce a new reward parameter which takes into account the load of each detected W-AP, the duration of the vertical handover, the offered gain, as well as the achieved signal-to-interference-plus-noise ratio. With the Q-learning scheme, each user decides to join the WiFi offloading or not, depending on the received reward from his environment and from his previous learning. In addition, since the AP's load value is very important in the reward parameter, an optimal value of the weight given to the channel load is solved under quality of service constraint. Simulation results showed the effectiveness of the proposed Q-learning-based scheme when compared with common WiFi offloading scheme in terms of cellular residence time.