Application of Efron's bootstrap methods to evaluate the performance of neural networks in the classification of mammographic features.

摘要

Efron's bootstrap resampling method is proposed to analyze the performance of artificial neural networks (ANNs) in the area of feature classification for the analysis of mammographic masses. The performance of ANNs is typically measured in terms of the area under the receiver operating characteristics (ROC) curve (Az). We investigate three methods for designing an ANN classifier, using "leave-one-out" cross-validation, bootstrapping ROC curve measures (AZ), and bootstrapping original sample set. Our experiments show that the performance of ANNs in terms of AZ is not a fixed value, but follows a distribution function. Nonparametric bootstrap methods can be used to accurately estimate the probability model and the characteristics of the performance of ANNs in a small-sample-size situation. The approach of training ANNs by bootstrapping the original sample set is shown to produce smaller bias and variance compared to the other two approaches.