Strategic Learning of Cross-layer Design for Channel Access and Transmission Rate Adaptation in Energy-constrained Cognitive Radio Networks

摘要

In this paper, we investigate the cross-layer strategic design of joint channel access and transmission rate adaptation in Cognitive Radio (CR) networks. Our target is minimizing the cost function which jointly considers the energy consumption in physical layer and the packet loss in data link layer. If the dynamic, time-varying nature of CR environment is completely known, the problem can be formulated as a Markov Decision Process (MDP). However, for the unknown CR environment, CR users should apply the multi-agent Reinforcement Learning (RL) to design the strategy for channel access and transmission rate choice. The multi-agent reinforcement learning is decentralized applied in the framework of Correlated Equilibrium (CE)-Q learning and the convergence is guaranteed. Furthermore, by setting different values for the parameter in cost function, we can adjust the tradeoff between energy efficient and the packet loss rate. Simulation results show the performances of multi-agent RL approach that of the MDP solution and the parameter in cost function can efficiently adjust the tradeoff of energy consumption and the packet loss rate.