A Bilinear Multi-Scale Convolutional Neural Network for Fine-grained Object Classification

摘要

Coarse-grained object classification with simple backgrounds has become quite mature, but the fine-grained object classification task against complex backgrounds is still challenging because there are only subtle differences in the local areas between different classes of fine-grained objects. In this paper, we propose a novel multi-scale convolutional neural network (msCNN) architecture that used for fine-grained object classification, which can extract discriminate local features at different scales in pyramid scale space. And a simplified bilinear model is used to carry out the end-to-end training for object classification. In the training phase, we design a distributed learning method with sample penalty term based on the sample distribution to optimize the network, which can improve the generalization ability of the network. In addition, we avoid using costly manual annotations like bounding box throughout the training and classification process. Finally, we present extensive experiments and visualizations on CUB-200-2011 dataset and ILSVRC2012_Dog dataset that analyze the effects of the bilinear msCNN model on the fine-grained object classification task. The classification accuracy shows that the significant improvement of our msCNN model on the fine-grained object classification.