A Chaotic Q-learning-Based Licensed Assisted Access Scheme Over the Unlicensed Spectrum

摘要

To meet the ever-increasing demand for mobile data traffic, mobile operators are seeking to utilize unlicensed spectrum as a supplement to the licensed spectrum. The harmonious and spectrum-efficient coexistence scheme between LTE and incumbent users on the unlicensed spectrum is thus mandatory. Currently, advanced intelligent technologies are being expected to play the crucial role in the future communication system. We thus introduce the Q-learning (QL) framework into LTE licensed assisted access (LAA) scheme in the paper, thereby forming a QL based LAA scheme. We first redefine the fairness in the sharing of unlicensed spectrum and then divide the state space into six states based on the predefined throughput and fairness thresholds, followed by the definition of the action set and reward function. In the proposed scheme, based on the convergent Q table, where each element is used to evaluate the pros and cons of taking an action, the agent can repeatedly interact with the environment until it reaches the terminal state, i.e. selects the optimal action (i.e. contention window size). Additionally, the chaotic motion with ergodicity, regularity and randomness is first introduced into the action-decision strategy to accelerate the training velocity with the balance consideration of exploration and exploitation. The simulation results prove that the proposed $epsilon$-chaotic greedy selection strategy has faster convergence velocity compared with other methods such as $epsilon$-greedy, pure greedy, Bolzmann and random selection strategy, and that the proposed chaotic QL LAA scheme outperforms the other LAA schemes such as the 3GPP, linear, fixed LAA and Listen Before Talk (LBT) adaptive schemes in terms of throughput, collision probability, fairness and delay.