PLACEMENT-AWARE ACCELERATION OF PARAMETER OPTIMIZATION IN A PREDICTIVE MODEL

摘要

Exemplary methods of optimizing a predictive model of a system using training data and knowledge of the placement of parameters within the predictive model comprises generating a set of slices comprising the predictive model topology, the set of slices comprising a primary slice, a first layer comprising subsets of the primary slice, and a second layer comprising subsets of first layer slices. The exemplary methods also comprise performing optimizations of the predictive model during one or more epochs, each involving dividing the training data into a plurality of batches, using the batches initially to train second-layer slices, then to train first-layer slices, and finally to train the primary slice. Optimized values determined for smaller slices and/or during previous epochs can be used to determine optimized values for larger slices (including primary slice) and/or later epochs. Embodiments include a parallel computing apparatus configurable to perform any of the exemplary methods.