Sample Size Determination for Image Classification Accuracy Assessment and Comparison

摘要

The classification accuracy statement is the basis of the evaluation of a classification's fitness for purpose.Accuracy statements are also used for applications such as the evaluation of classifiers,with attention focused especially on differences in the accuracy with which data are classified.Many factors influence the value of a classification accuracy assessment and evaluation programme.This paper focuses on the size of the testing set(s),and its impacts on accuracy assessment and comparison.Testing set size is important as an inappropriately large or small sample could lead to limited and sometimes erroneous assessments of accuracy and of differences in accuracy.In this paper the basic statistical principles of sample size determination are outlined.Some of the basic issues of sample size determination for accuracy assessment and accuracy comparison are discussed.With the latter,the researcher should specify the effect size (minimum meaningful difference),significance level and power used in an analysis and ideally also fit confidence limits to estimates.This will help design a study as well as aid interpretation.In particular,it will help avoid problems such as under-powered analyses and provide a richer information base for classification evaluation.Central to the argument is a discussion of Type II errors and their control.The paper includes equations that could be used to determine sample sizes for common applications in remote sensing,using both independent and related samples.