Informative Deep Reinforcement Path Planning for Heterogeneous Autonomous Surface Vehicles in Large Water Resources

Abstract

Water contamination in extensive aquatic resources is a pressing issue, especially during current drought conditions across the world. To adress this, a novel approach involving a heterogeneous sensing capabilities fleet of four autonomous surface vehicles is introduced for efficient contamination mapping. To reduce costs, vehicles may be equipped with low quality sensors meaning measurements reliability differs between vehicles and affects model accuracy. The diverse sensing capabilities are characterized by a wide range of sensor standard deviations, addressing the applicability of the framework in real-world scenarios with commercial sensors. This research leverages Gaussian Processes to accurately model spatial distribution of contamination, integrating measurements from the vehicles to understand contamination patterns comprehensively. Additionally, an informative path planning strategy is introduced based on a centralized neural network which implements a Double Deep Q-Learning algorithm, driving the decision-making process of all agents. Effective learning hinges on accurately defining the observation and reward functions, for which several proposals will be compared. These tailored definitions are essential for guiding the learning process, and minimizing the error towards the main goal: to obtain the best possible contamination model. Remarkably, the proposed system demonstrates superior performance in Ypacaraí Lake scenario, surpassing traditional heuristics like lawn mower or particle swarm optimization by up to 82% in reducing mean squared error in highly contaminated regions for several combinations of agents.