Visualisation of Measures of Classifier Reliability and Error in Remote Sensing

摘要

The estimation of the accuracy of a thematic classification derived from remotely sensed data is generally based on the confusion or error matrix. This matrix is derived by evaluating the performance of the classifier on a set of test data, and the quantities derived from analysis of the confusion matrix include percent accuracy (total and per class), producer's accuracy, consumer's accuracy, and varieties of the kappa coefficient. None of these measures considers the spatial distribution of erroneously classified pixels, either implicitly or explicitly. Furthermore, each pixel in the image is assigned a unique ("hard") label but generally no measure of confidence is assigned to that label. In this paper we propose a methodology that specifically takes into account the spatial pattern of errors of omission and commission, and which presents the user with an indication of the reliability of pixel label assignments. Our methodology assumes that an accurate digital map of the spatial objects (fields, lakes, or forests) being classified, plus cultural features such as roads and urban areas, is available. Such a map can be obtained by digitising a large-scale paper map of the study area, or via image processing. Assuming also that the number of classes is appropriate to the problem at hand and to the scale of the image, it is likely that a substantial number of erroneous label allocations relate to mixed pixels located near or on field boundaries. A buffering operation is applied to the digitised boundary information in order to generate a mask. The width of the buffer is determined from a study of the location of the erroneously labelled pixels. To this mask are added regions not included in the classification, for example roads and urban areas. Furthermore, we look at the spatial distribution of the remaining errors to determine whether these errors are spatially random or clustered in their distribution. Such information can help to refine the classification. Finally, we present a method using colour coding that allows the visualisation of the reliability of the classifier output. The method can be applied to neural or statistical classifiers.