A Fast Robustness Quantification Method for Evaluating Typical Deep Learning Models by Generally Image Processing

摘要

Deep learning especially image recognition techniques have been extensively used in various applications such as unmanned driving. The robustness of deep learning models is of critical importance since fault image recognition results may result in serious incidents. In this paper, we propose a fast quantifying method by general image processing to evaluate the robustness of five typical deep learning models under the Keras framework (i.e., VGG16, InceptionV3, ResNet50, DenseNet, and MobileNet). We analyze six metrics in terms of accuracy, precision, recall, F1, recognition time, and impact factor. The evaluation data is publicly accessible image data sets from Kaggle. In our evaluation, the adversary samples are generated by generally image processing methods such as gray-scaling, color-reversing, and image-flipping, which is ordinary operations and easily launched. The different models over various image processing methods are evaluated and compared comprehensively. The evaluation results show that DenseNet performs best over three conditions such as baseline, grayscaling and horizontal flipping. MobileNet costs the shortest delay in decision over all image processing methods. F1 score varies with different attack intensity. InceptionV3 presents overall robustness in most conditions.