Evaluating assisted target recognition performance: an assessment of DARPA's SAIP system

摘要

New advanced imaging systems will soon be capable of collecting enormous volumes of imagery, placing a significant burden on the imagery analysts (IAs) that exploit these data. ATRs and other image understanding tools offer a way to assist IAs in exploiting large volumes of imagery more effectively and efficiently. The Defense Advanced Research Project Agency (DARPA) Semi-Automated IMINT Processing (SAIP) Program focuses on these technologies to assist IAs in the timely exploitation of SAR imagery. The SAIP system is an integrated set of imagery exploitation tools designed to improve the capability of the IA to support military missions in a tactical environment. To assess the utility of the SAIP technology, a mix of live and playback exercises were conducted. IAs exploited the imagery with the assistance of the SAIP technology. As a benchmark for comparison, the same imagery was exploited in an operational exploitation system without the benefit of SAIP assistance. This paper presents the methodology for assessing exploitation performance and discusses issues related to scoring exploitation performance. The results of a recent assessment event illustrate the issues and provide guidance for future work in this area.