A hybrid method based on estimation of distribution algorithms to train convolutional neural networks for text categorization

摘要

? 2022 Elsevier B.V.Convolutional Neural Networks for text categorization allows the extraction of features from the text represented through word embedding. The high dimensionality of the texts themselves implies a larger number of network parameters and a more complex optimization surface. Artificial neural network training is an NP-Hard optimization problem, which has been addressed by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minimums. In this work, we propose a hybrid method based on the Estimation of Distribution Algorithms for training a Convolutional Neural Network. For this, we train together gradient-based methods with the Estimation of Multivariate Normal Algorithm and Univariate Marginal Distribution Algorithm by dividing the training process into two stages. The different variants obtained with the proposed method are compared with gradient-based methods on public benchmark datasets and statistical differences are analyzed by nonparametric tests. The proposed method increases the accuracy of the convolutional network applied to the text categorization task and overcome in about 0.22–24 the state-of-the-art algorithms.