A Tikhonov Approach to Calculate Regularisation Matrices

摘要

Regularisation is a popular method to overcome the ill-conditioning learning problems in neural networks. This is achieved by penalizing the performance criteria adding some prior distribution on the weights, usually a quadratic weight function, i.e. E~d = w~tKw. Find out the regularisation matrix K included in the regularisation solution of learning problems, is a difficult and computationally expensive task [2]. This paper provide an efficient and easy way for K matrix calculation and the development of this method for Zero and Second Order Regularisation.