Towards the robustness in neural network training

摘要

Though proven to be very successful in many cases where other traditional techniques failed to give satisfactory results, neural networks still raise a lot of questions. Disbelief comes from difficulties with correct choice of network parameters, like initial set of weights, adequate network architecture, etc. The proposed method uses combination of two different approaches: genetic algorithm and gradient method approach. The proposed approach automatically searches for the adequate initial weight set. The robustness with respect to initial weight set is achieved through introduction of randomness in neuron weight space. Process goes as following. Genetic approach is used in process of searching for weight set with minimal total error. Once that set is determined, algorithm uses the second, gradient type of approach. The proposed algorithm is not based on typical gradient type of search, rather it estimates the gradient from series of feed forward calculations. Results are confirmed through experimental data and given in form of graphs.