Empirical Evaluation of the Virtual Autonomous Navigation Environment

摘要

The US Army Corps of Engineers' (USACE) Virtual Autonomous Navigation Environment (VANE) is a physics-based, multi-scale numerical testbed designed to quantitatively and accurately predict sensor and autonomous system performance in a simulation environment. The work presented here captures progress on an initial empirical evaluation of how well the current VANE system is able to reproduce a real autonomy system's perception performance. Findings will directly guide continuing development of VANE, while beginning to develop a suite of example sensor models and virtual environments. This first experiment focuses on testing world modeling and sensor simulation. Data was collected from the Crusher autonomous vehicle, developed under the DARPA UPI program. Some sensor data was collected and manually processed to produce a VANE scene model. Crusher was again driven through the real scene to collect real sensor data as the baseline sensor data. The positions of the sensors were extracted and was used to generate a VANE simulation to exactly match Crusher's path. Both datasets were fed to an offline version of Crusher's autonomous perception software. The outputs from the two separate input data sets were compared. The results indicate good agreement between the outputs, especially on solid ground and solid objects. Differences were observed in the areas of vegetation, an area requiring further work to improve modeling and simulation of the sensors. Greater accuracy will also be required in the ground truth data, which was collected at WAAS GPS quality rather than RTK2 quality.