A Survey and Performance Evaluation of Reinforcement Learning Based Spectrum Aware Routing in Cognitive Radio Ad Hoc Networks

摘要

Cognitive radio technology is an assuring solution for under-utilization of licensed spectrum bands and overcrowding of unlicensed spectrum bands, in which secondary user is permitted to access the primary users' spectrum in an opportunistic manner. Opportunistic access of the spectrum requires complex changes across all the layers of a network protocol stack. Cognitive radio has to be an autonomous agent in order to configure itself to dynamic spectrum environment. And, the characteristics of reinforcement learning, a subfield of artificial intelligence in which the agent learns the surrounding operating environment through continuous interaction and takes an optimum decision on the fly, is in compliance with features of self-organized cognitive radio ad hoc network. Therefore, reinforcement learning is an appropriate option for incorporating intelligence and self-adaptivity into cognitive radio. This paper provides a comprehensive survey on the application of reinforcement learning for efficient spectrum aware routing in cognitive radio ad hoc network. The preliminaries of cognitive radio ad hoc networks and reinforcement learning are first introduced, and a review is investigated in the proposed research area along with a discussion on open research challenges with an aim to promote research. From the survey, reinforcement learning incorporated cognitive radio can learn the unknown primary user network model and the learned model can be then used for finding a suitable route to meet the Quality of Service requirements. With this in mind, the paper also proposes a multi-objective reinforcement learning based spectrum aware routing protocol with an aim to maximize the probability of successful transmission using a minimum hop path. The simulated results prove the performance of the algorithm.