Probability Indexes of Image Interpreter Performance: Development and Evaluation

摘要

The study was conducted in a two-phase project to explore the development of indexes predictive of interpretation results and the effect on performance of probabilistic feedback given interpreters while they are examining imagery. Statistical treatment and analysis of data, based on the judgment and performance of trained interpreters, is described in detail. The ten best predictors of accuracy of target identification and the eight best predictors of the value of continued search (search utility) were identified and their predictive validity assessed in a three-session experiment. The predictors contributing most to the validity for accuracy of performance were (1) judged target difficulty and (2) interpreter's confidence in his identifications. Predictors of search utility were of generally low validity. More effective prediction was obtained for target identification than for target detection or classification. It was found that feedback of both accuracy and search utility probability data reduced the number of incorrect identifications. Overall individual accuracy, completeness, or rate of interpretation was not affected, however, by feedback. Based on the findings in this study, it seems feasible that interpreters' accuracy predictions can be substituted for confidence ratings now used as more precise indications of dependability of the information. Predictions have some utility also in weighting indentifications made by several interpreters or in conjunction with relative estimated cost errors--omissions vs misidentifications--in selecting among response alternatives within an image interpretation facility. (Author)