Weighted cooperative reinforcement learning-based energy-efficient autonomous resource selection strategy for underlay D2D communication

摘要

Underlay Device-to-Device (D2D) communication is a key technology responsible for high data rate, ultra-low latency with high spectral and energy efficiency in 5G cellular networks. But to achieve its full potential, optimal channel allocation and effective co-channel interference management must be accomplished. To address this challenge, we propose a multi-agent reinforcement learning based autonomous channel selection scheme for D2D communication. The proposed scheme, Weighted Cooperative Q-Learning based Resource Selection (WCopQLRS), allows a D2D pair to learn to select a channel from the available resources autonomously. Learning process of each D2D transmitter involves cooperation from neighboring D2D agents by exchanging their latest Q-values. An additional parameter called cooperation range is used to determine the neighboring pairs whose Q-values can be used for learning the optimal policy. The limited prior information prevents a linear increase in the dimensions of Q-value matrix of each learning agent when the number of D2D pairs within the cell is huge. Though WCopQL-RS involves additional information exchange among agents as compared to independent learning but also provides improved system throughput and convergence speed. It is shown through simulation results that WCopQL-RS outperforms other existing schemes in terms of average D2D user throughput, energy consumption and fairness value.