Probabilistic based recursive model for adaptive processing of data structures

摘要

One of the most popular frameworks for the adaptive processing of data structures to date, was proposed by Frasconi et al. [Frasconi, P., Gori, M., & Sperduti, A. (1998). A general framework for adaptive processing of data structures. IEEE Transactions on Neural Networks, 9(September), 768-785], who used a Backpropagation Through Structures (BPTS) algorithm [Goller, C, & Kuchler, A. (1996). Learning task-dependent distributed representations by back-propagation through structures. In Proceedings of IEEE international conference on neural networks (pp. 347-352); Tsoi, A. C. (1998). Adaptive processing of data structure: An expository overview and comments. Technical report in Faculty Informatics. Wollongong, Australia: University of Wollongong] to carry out supervised learning. This supervised model has been successfully applied to a number of learning tasks that involve complex symbolic structural patterns, such as image semantic structures, internet behavior, and chemical compounds. In this paper, we extend this model, using probabilistic estimates to acquire discriminative information from the learning patterns. Using this probabilistic estimation, smooth discriminant boundaries can be obtained through a process of clustering onto the observed input attributes. This approach enhances the ability of class discrimination techniques to recognize structural patterns. The proposed model is represented by a set of Gaussian Mixture Models (GMMs) at the hidden layer and a set of "weighted sum input to sigmoid function" models at the output layer. The proposed model's learning framework is divided into two phases: (a) locally unsupervised learning for estimating the parameters of the GMMs and (b) globally supervised learning for fine-tuning the GMMs' parameters and optimizing weights at the output layer. The unsupervised learning phase is formulated as a maximum likelihood problem that is solved by the expectation-maximization (EM) algorithm. The supervised learning phase is formulated as a cost minimization problem, using the least squares optimization or Levenberg-Marquardt method. The capabilities of the proposed model are evaluated in several simulation platforms. From the results of the simulations, not only does the proposed model outperform the original recursive model in terms of learning performance, but it is also significantly better at classifying and recognizing structural patterns.