Scaling complex models for neural networks

摘要

A method for determining the final architecture for a neural network to perform a specific machine learning task is described. The method includes receiving a basic architecture for a neural network, the basic architecture having a network width dimension, a network depth dimension and a resolution dimension; receiving data to define complex coefficients that control additional computational resources used to scale the underlying architecture; performing a search to determine base width, depth, and resolution coefficients, respectively, specifying how to allocate additional computational resources to the network width, depth, and resolution dimensions of the underlying architecture; determining width, depth, and resolution coefficients based on the base width, depth, and resolution coefficients and the composite coefficient; and generating a final architecture that scales the network width, network depth and resolution dimensions of the base architecture based on the corresponding width, depth and resolution coefficients.