Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks

摘要

Channel handoff is a crucial function for Cognitive radio ad hoc networks (CRAHNs). The ab-sence of centralized infrastructures and the limited power make the handoff design more challenging. A learning-based interference-aware handoff scheme is proposed for distributed CRAHNs. We model the channel handoff pro-cess as a Partially observable Markov decision process (POMDP) and adopt a Q-learning algorithm to find an optimal handoff strategy in a long term. The proposed algorithm obtains an efficient transmission performance by considering the interferences among SUs and PUs. To achieve PU awareness, the handoff scheme predicts the PU activities by using the historical channel usage statistics. In addition, we also propose a refined channel selection rule to compromise between learning speed and cumulative trans-mission reward. The simulation results show that the pro-posed handoff scheme can adapt to the PU activities and achieves a better performance in terms of high throughput and low collisions. And the learning process keeps a consid-erable balance between convergence time and cumulative reward.