Assessment and comparison of two pre-training methods for regularizing deep architecture neural networks in face recognition

摘要

Training deep architecture neural networks have been considered widely recently. Besides the common and direct approach of training neural networks in which gradient-based training is used after choosing the appropriate structure, new approaches based on pre-training are more efficient in learning highly-varying complex functions. Not only gradient-based training is very slow, but also due to starting from random initialization, they generally get stuck in apparent local minimum and as the architecture gets deeper, it becomes more difficult to obtain good generalization. One way to solve this problem is to use pre-training and the multi-layer network is broken into some learning machines with the purpose of finding a good starting point in the function space of network. In this paper two pre-training methods are applied to obtain a suitable region of initial weights which is better than random initialization. Both two methods are implemented and used in a face recognition task on AT&T database. Results show that although first pretraining method is faster, the second method leads to better generalization and higher recognition rate.