A novel companion objective function for regularization of deep convolutional neural networks

摘要

Regularization is an essential technique discussed in an attempt to solve the overfitting problem in deep convolutional neural networks (CNNs). In this paper, we proposed a novel companion objective function as a regularization strategy to boost the classification performance in deep CNNs. Three aspects of this companion objective function are studied. Firstly, we proposed two kinds of auxiliary supervision for convolutional filters and non-linear activations respectively in the companion objective function. Both of them enhanced the performance by aleviating the overfitting problem and auxiliary supervision for non-linear activations provided more efficiency. Secondly, regularization of auxiliary supervision in the pre-training phrase is discussed. With the assistance of auxiliary supervision, CNNs could obtain a more favorable initialization for end-to-end supervised fine-tuning. Finally, this companion objective function is verified to be compatible with other regularization strategies such as dropout and data augmentation. Experimental results on benchmark datasets (CIFAR-10 and CIFAR-100) demonstrated advantages of our proposed companion objective function as a regularization approach. (C) 2016 Published by Elsevier B.V.