Reinforcement Q-Learning applied to underwater search planning towards maximizing information gain in environments with variable target detection probabilities

摘要

The objective of an autonomous underwater vehicle (AUV) in an exploration mission is to maximize its information gathering about the environment it explores. In a mine countermeasures (MCM) survey the goal is to detect and localize targets. The work presented addresses the efficacy of reinforcement Q-Learning-based planning to localize targets in an area where the sonar signal-to-noise-ratio (SNR) may vary, and consequently, the probability of detection and the likelihood of false alarm may vary as well. A sensitivity study of the proposed algorithm is provided, which identifies the parameters that drive the planner’s performance. A visual example is used to illustrate the parameters’ impact on the path-planning policy. Results from simulation and two-dimensional experimentation are presented for MCM missions in two different search areas with varying environmental conditions.