An adaptive stepsize plus integral controller for improvement to the steepest descent algorithm

摘要

The classical approach to steepest-descent optimization is viewed from a different perspective, namely, control-systems in the frequency-domain. Using classical control methods, it becomes possible to view the optimization problem as a control-system and then to implement a controller to improve the algorithm so as to achieve a lower error. An adaptive stepsize proportional plus integral algorithm is derived to cope with time-varying problems. For such problems the gain is time varying and the closed-loop system must be adaptive by varying the loop gain to maintain stability.