A Structural Learning Method of Restricted Boltzmann Machine by Neuron Generation and Annihilation Algorithm

摘要

Restricted Boltzmann Machine (RBM) is a generative stochastic energy-based model of artificial neural network for unsupervised learning. The adaptive learning method that can discover the optimal number of hidden neurons according to the input space is important method in terms of the stability of energy as well as the computational cost although a traditional RBM model cannot change its network structure during learning phase. Moreover, we should consider the regularities in the sparse of network to extract explicit knowledge from the network because the trained network is often a black box. In this paper, we propose the combination method of adaptive and structural learning method of RBM with Forgetting that can discover the regularities in the trained network. We evaluated our proposed model on MNIST and CIFAR-10 datasets.