Method of digging tunnels horizontally into the error hypersurface to speed up training and to escape from local minima

摘要

In this paper, a general compressing method is reviewed systematically at first. Then the idea and steps of digging horizontally into the error hypersurface are presented, as well as an example. Since there exists serious and complex nonlinearity in the error hypersurface, training by gradient descending techniques is often too slow when it is on a plateau and has the risk of trapping into local minima. Digging tunnels into the error hypersurface by means of rotation transformation will lead to the plateau to speed up the training or skip from local minima.