Sequential and Parallel Methods for Unconstrained Optimization

摘要

This paper reviews some interesting recent developments in the field of unconstrained optimization. First we discuss some recent research regarding secant (quasi-Newton) methods. This includes analysis that has led to an improved understanding of the comparative behavior of the BFGS, DFP, and other updates in the Broyden class, as well as computational and theoretical work that has led to a revival of interest in the symmetric rank one update. Second we discuss recent research in methods that utilize second derivatives. We describe tensor methods for unconstrained optimization, which have achieved considerable gains in efficiency by augmenting the standard quadratic model with low rank third and fourth order terms, in order to allow the model to interpolate some function and gradient information from previous iterations. Finally, we will review some work that has been done in constructing general purpose methods for solving unconstrained optimization problems on parallel computers. This research has led to a renewed interest in various ways of performing the linear algebra computations in secant methods, and to new algorithms that make use of multiple concurrent function evaluations. (kr)