互学习神经网络训练方法研究

摘要

由于BP神经网络具有表达能力强,模型简单等特点,经过近30年的发展,在理论和应用研究上都取得了巨大的进步,然而容易陷入局部最优和泛化能力差等问题却限制了神经网络的发展.同时,大数据的出现和深度学习算法的提出与应用,为神经网络向更类脑的方向发展提出了新的要求.针对上述问题,该文从模拟生物双向认知能力的角度出发,构造了一种新的神经网络模型——互学习神经网络模型,该模型在标准正向神经网络的基础上,引入了与其具有结构对称性的负向神经网络,利用正、负向神经网络分别模拟生物的顺向和逆向认知过程,并在此基础上提出了一种新的神经网络训练方法——互学习神经网络训练方法,该方法通过网络连接权值转置共享,正、负双向交替训练的方式对互学习神经网络模型进行训练,从而实现输入数据和输出标签之间的相互学习,使网络具有双向认知能力.实验表明,互学习神经网络训练方法可以同时训练正、负两个神经网络,并使网络收敛.同时,在此基础上提出了"互学习预训练+标准正向训练"的两阶段学习策略和相应的转换学习方法,这种转换学习方法起到了和"无监督预训练+监督微调"相同的效果,能够使网络训练效果更好,是一种快速、稳定、泛化能力强的新型神经网络学习方法.%Since BP neural network is expressive and model is simple,there has been a great improvement in both theoretical and applied research over the past 30 years.But its development is held back due to model limitations on local optimism and overfitting.With the emergence of big data and application of deep learning,there are new requirements which gears neural network development towards more pseudo brain.To solve the above problem,this paper presents a new neural network model based on the simulation of biological bidirectional cognitive ability: the mutual learning neural network model.The design of the mutual learning neural network model originates from human being''s bidirectional cognitive ability,that is,the forward cognitive ability and the backward cognitive ability,and the previous one possesses the cause and demands the result while conversely the latter one possesses the result and demands the cause.The mutual learning neural network model is composed of the positive neural network and the negative neural network.The positive neural network is a feedforward neural network with a hidden layer,which is used to set up the cognitive relationship from the cause(data)to the result(label)and simulate the forward cognitive ability.The negative neural network has the symmetrical relationship with the positive neural network,and it''s mainly used to set up the cognitive relationship from the result(label)to the cause(data)and simulate the backward cognitive ability.These two neural networks are combined together by weight sharing and construct the neural network model together,which simulates human being''s bidirectional cognitive procedure.Based on it,this paper proposes a new training method of neural network: mutual learning neural network training method.Firstly,the input of the mutual learning neural network training method is the data and output of it is the label,which trains the positive neural network through BP learning algorithm.After a certain times of training,the forward link weight matrix is updated and the value assigned to it is transported to the negative neural network(the bias term is independent of each other).Then use the label to input and the data to output,and train the negative neural network by BP learning algorithm.After certain times of training,the backward link weight matrix is also updated and the value assigned to it is transposed to the positive neural network(the bias term is independent of each other).Such reciprocate alternation is continued until the end of iteration.So the mutual learning neural network training method realizes the mutual learning procedure between the input of the data and the output of the label,and by training it enables the mutual learning neural network model possess the bidirectional cognitive ability.Experiment results show that the mutual learning neural network training method can train both positive and negative network simultaneously,and it is a convergent learning algorithm.In addition,this paper also proposes the "mutual learning neural network training+standard positive neural network training",a two stage learning strategy,making it as effective as "pre training+ fine-tuning" learning strategies,thus,making the network training more effective.This is a fast,stable,and widely generalized neural network training method.