TRAINING OPTIMIZATION FOR NEURAL NETWORKS WITH BATCH NORM LAYERS

摘要

In an embodiment, a method includes training a neural network model with a first set of training data. In an embodiment, the method includes calculating divergence for a set of layers of the neural network model, the set of layers comprising at least one batch norm layer. In an embodiment, the method includes analyzing, based on the calculated divergence, a stability of each of the set of layers. In an embodiment, the method includes removing, based on the analysis determining a subset of the set of layers fails to meet a threshold stability, the subset of the set of layers of the neural network model.