ADVANCED NEURAL NETWORK LEARNING APPLIED TO MODELING OF PULPING OF SUGAR MAPLE

摘要

This paper reports work done to improve the modeling of complex processes when only small experimental datum sets are available. Neural networks are used to capture the nonlinear underlying phenomena contained in the data set and to partly eliminate the burden of having to specify completely the structure of the model. Two different types of neural networks were used for the application of Pulping of Sugar Maple problem. A three layer feed forward neural networks, using the Preconditioned Conjugate Gradient (PCG) methods were used in this investigation. Preconditioning is a method to improve convergence by lowering the condition number and increasing the eigenvalues clustering. The idea is to solve the modified problem M~(-1) Ax = M~(-1) b where Mis a positive-definite preconditioner that is closely related to A. We mainly focused on Preconditioned Conjugate Gradient- based training methods which originated from optimization theory, namely Preconditioned Conjugate Gradient with Fletcher-Reeves Update (PCGF), Preconditioned Conjugate Gradient with Polak-Ribiere Update (PCGP) and Preconditioned Conjugate Gradient with Powell-Beale Restarts (PCGB). The computational experiments revealed that the PCG methods, through its preconditioner matrix, reduced drastically the mean squared error during training. The CG and PCG methods have a much faster convergence rate than the BP; since it uses second order information to calculate the new direction (Rao, 1978). All the trials for the PCG methods converged to the required solution indicating that the choice of initial weights is appropriate. Even though it does not guarantee convergence to the required mean squared error, it reduces the possibility of getting stuck in local minima. The PCG algorithms have proved its capability in the generalization aspect, at the same time preserving the efficient convergence of the algorithm. The variation in the network outputs is explained very well by the corresponding targets. The results of the simulations suggest that, using preconditioning techniques, the condition numbers of a matrix will be improved. If the eigenvalues of the input matrix have been clustered, we can iteratively solve the problem more quickly. The behavior of the PCG methods in the simulations proved to be robust against phenomenon such as oscillations due to large step size.