Comparing different methods for assessing ground truth of rover data analysis for the 2005 season of the Life in the Atacama Project

摘要

The scientific success of a remote exploration rover mission depends on the right combination of technology, teamwork and scientific insight. In order to quantitatively evaluate the success of a rover field trial, it is necessary to assess the accuracy of scientific interpretations made during the field test. This work compares three structured approaches to assessing the ground truth of scientific findings from a science team conducting a remote investigation of a locale using an autonomous rover. For the first approach, independent assessment, the daily science summaries were analyzed and reduced to a series of 1082 factual statements, which were treated as hypotheses. An independent scientist traveled to the field area to assess these hypotheses. For the second approach, guided self-study, the mission scientists themselves traveled to the field area and evaluated their own scientific interpretations. The third approach, discrepancy investigation, searched for the root causes of differences between the scientific interpretations made in the control room and those made in the field. The independent investigation provided sensitive, quantitative data, but suffered from the lack of context and continuity developed in the mission control room. The guided evaluation benefited from the context of the mission, but lacked clarity and consistency. The discrepancy investigation provided insight into the root causes behind the discrepancies, but was expensive and time consuming. The independent investigation method yielded particularly compelling results, but each method offers advantages and a comprehensive rover field trial assessment should include a combination of all three.