Self-monitoring Reinforcement Metalearning for Energy Conservation in Data-ferried Sensor Networks

摘要

Given multiple widespread stationary data sources such as ground-based sensors, an unmanned aircraft can fly over the sensors and retrieve their data via a wireless link. When sensors have limited energy resources, they can reduce the energy used in data transmission if the ferry aircraft is allowed to extend its flight time. Complex vehicle and communication dynamics and imperfect knowledge of the environment confound planning since accurate system models are difficult to acquire and maintain, so we present a reinforcement learning approach that allows the ferry aircraft to optimise data collection trajectories and sensor energy use in situ, obviating the need for system identification. We address a key problem of reinforcement learning-the high cost of acquiring sufficient experience-by introducing a metalearner that transfers knowledge between tasks, thereby reducing the number of flights required and the frequency of significantly suboptimal flights. The metalearner monitors the quality of its own output in order to ensure that its recommendations are used only when they are likely to be beneficial. We find that allowing the ferry aircraft to double its range can reduce sensor radio transmission energy by 60% or better, depending on the accuracy of the aircraft's information about sensor locations.