UAV Networks Against Multiple Maneuvering Smart Jamming With Knowledge-Based Reinforcement Learning

摘要

The unmanned aerial vehicles (UAVs) networks are very vulnerable to smart jammers that can choose their jamming strategy based on the ongoing channel state accordingly. Although reinforcement learning (RL) algorithms can give UAV networks the ability to make intelligent decisions, the high-dimensional state space makes it difficult for algorithms to converge quickly. This article proposes a knowledge-based RL method, which uses domain knowledge to compress the state space that the agent needs to explore and then improve the algorithm convergence speed. Specifically, we use the inertial law of the aircraft and the law of signal attenuation in free space to guide the highly efficient exploration of the UAVs in the state space. We incorporate the performance indicators of the receiver and the subjective value of the task into the design of the reward function, and build a virtual environment for pretraining to accelerate the convergence of anti-jamming decisions. In addition, the algorithm proposed is completely based on observable data, which is more realistic than those studies that assume the position or the channel strategy of the jammer. The simulation shows that the proposed algorithm can outperform the benchmarks of model-free RL algorithm in terms of converge speed and averaged reward.