Q-Learning-Based Cross-Layer Learning Engine Design for Cognitive Radio Network

摘要

In cognitive radio (CR) networks, Learning Engine has considerable significance on dynamic spectrum access (DSA) and implementation of cognitive function. In this paper, a cross-layer learning engine design scheme is proposed by jointly considering physical-layer dynamic channel selection, modulation and coding scheme, data-link layer frame length in CR networks, with the purpose to maximize system throughput and simultaneously meet heterogeneous Quality of Service (QoS) requirements. The wireless fading channel is modeled as a continuous state space Markov decision process (MDP) and the licensed network activity is abstracted as a finite-state one. We introduce Q-learning algorithm to realize the function of learning from state space and adapt wireless environment. And meanwhile a large scale Q-function approximator based on support vector machine (SVM) is employed to effectively reduce storage requirement and decrease the operation complexity. A cross-layer learning engine communication platform is realized by using Matlab simulator, the simulation results demonstrate that while lacking system prior knowledge, the learning engine can effectively achieve configuration function by system cross-layer learning approach, and furthermore, it can converge to the best-i.e., realize reconfiguration function in CR networks while meeting users' QoS.