Neural Network Model Switching for Efficient Feature Extraction

摘要

In order to improve the efficiency of the feature extraction of backpropagation (BP) learning in layered neural networks, model switching for changing the function model with- out altering the map is proposed. Model switching involves map preserving reduction of units by channel fusion, or addition of units by. channel installation. For reducing the model size by channel fusion. two criteria for detection of the redundant chan- nels are addressed, and the local link weight compensations for map preservation are formulated. The upper limits of the dis- crepancies between the maps of the switched models are derived for use as the unified criterion in selecting the switching model candidate. In the experiments, model switching is used during the BP training of a layered network model for image texture classification, to aid its inefficiency of feature extraction. The results showed that fusion and re-installation of redundant chan- nels, weight compensations on channel fusion for mal preserva- tion, and the use of the unified criterion for model selection areall effective for improved generalization ability and quick learning. Further, the possibility of using model switching for concurrent optimization of the model and the map will he discussed.