Global optimization of neural network

摘要

The local optimization problem of cost functions has blocked the advancement of not only the traditional mathematics but also the emerging Neural Network (NN). Since most of the cost functions used in the gradient descent of NN is non-convex, there are high possibilities that the algorithm may converge at the local optimum. In order to eliminate the substantial error caused from this problem, deep learning has been introduced. Yet, standard computing environment cannot possibly power calculation that deep learning requires. The fake alpha algorithm, newly introduced in this paper, shares the same goal with the deep learning algorithm, but can successfully be operated in standard computers. Additionally, the heuristic approach that the fake alpha algorithm takes in the gradient descent process enhances the efficiency of NN.