Speeding up training of deep neural networks using inconsistent stochastic gradient descent

摘要

Aspects of the present disclosure describe techniques for training convolutional neural networks using an inconsistent stochastic gradient descent (ISGD) algorithm. The training effort of the training batch used by the ISGD algorithm is dynamically adjusted according to the loss determined for the specific training batch and is subordinate to two sub-states (fully trained or well trained) Not). The ISGD algorithm increases iterations for batches that are not fully trained while reducing iterations for well trained batches.