NON-CONVEX OPTIMIZATION BY GRADIENT-ACCELERATED SIMULATED ANNEALING

摘要

A computer-implemented method of training a neural network comprises: identifying, by one or more processors, a set of parameters for a cost function for the neural network; applying, by the one or more processors, a gradient optimization of the cost function to generate a set of intermediate optimized parameter values for the cost function; initializing, by the one or more processors, a simulated annealing optimization of the cost function using the set of intermediate optimized parameter values for the cost function; applying, by the one or more processors, the simulated annealing optimization of the cost function to generate a set of final optimized parameter values for the cost function; assigning, by the one or more processors, the set of final optimized parameter values as weights for connections of the neural network; and using, by the one or more processors, the neural network to perform operations.