腫?識別器のLeave-One-Outによる性能評価結果の信頼性に関する考察

摘要

We report on the reliability of Leave-One-Out (L00) cross validation of a two class classifier, which classifies lesion images from false positive ones. It is known that the LOO is an unbiased estimator of generalization errors of classifiers but that it cannot estimate the deviations of the estimated errors from the true ones. In this article, we study on estimation of the deviations based on a uniform stability. We firstly demonstrate the relationship between the uniform stability and the deviation between the estimated error and the true one in simulated experiments. It was shown that the deviation became larger when the numbers of training samples of two classes were unbalanced and that the uniform stability could capture such the relationship. Secondly, we experimentally show the relationship between the uniform stability and the dimension of input data based on simulated experiments and clinical PET/CT images. It was experimentally found that the estimated error of a classifier could be improved only by increasing the dimension of input data but that the stability of the classifier became worse by the increase of the dimension.