Pruning Using Parameter and Neuronal Metrics

摘要

In this article, we introduce a measure of optimality for architecture se- lecition algorithms for neural networks: the distance from the original network to the new network in a metric defined by the probability distri- butions of all possible networks We derive two pruning algorithms, one based on a metric in parameter space and the other based on a metric in neuron space, which are closely related to well-known architecture selec- tion algorithms, such as GOBS. Our framework extends the theoretically range of validity of GOBS and therefore can explain results observed in previous experiments. In addition, we give some computational improve- ments for these algorithms.