Unmanned surface vessel obstacle avoidance with prior knowledge-based reward shaping

摘要

Autonomous obstacle avoidance control of unmanned surface vessels (USVs) in complex marine environments is always fundamental for its scientific search and detection. Traditional methods usually model USV motion and environments in a mathematical way that needs perceptual information. Unfortunately, it is difficult to provide sufficient perceptual information due to complex marine environments, resulting in inaccurate modeling. Reinforcement learning has recently enjoyed increasing popularity in the problem of obstacle avoidance since it can settle problems by partially observable environment information. However, the autonomous USV obstacle avoidance using reinforcement learning is still facing the challenge of designing appropriate reward functions under complex marine environments. To address these issues, we propose a prior knowledge-based USV reinforcement learning obstacle avoidance algorithm. In this algorithm, an actor-critic network is used as the main architecture of the algorithm, and prior knowledge-based reward shaping used to design relevant reward function for USV obstacle avoidance. A standard USV based on a visual sensor is designed, and the state input of the algorithm is through USV's front vision sensor. We conducted simulation experiments and results prove that our algorithm can effectively converge, and USV achieves high velocity and low collision rate in the complex marine environment.