Generalized Levenberg-Marquardt neural nets for minimization of quasiconvex scalar functions

摘要

Neural nets that minimize quasi-convex scalar functions are designed as dynamical systems (ordinary differential equations) which correspond to various well known discrete time algorithms, such as steepest descent, Newton, Levenberg-Marquardt, etc. The main contribution is a generalization of the Levenberg-Marquardt algorithm, including an adaptive version, that combines good features of the Newton and Levenberg-Marquardt algorithms and leads to trajectories that converge faster to the minimum of a quasiconvex objective function that is assumed to have known gradient and Hessian.¹