A geometric view of neural networks using homotopy

摘要

A homotopy approach is formulated for solving for the weights of a network. It is shown how this leads simply to a geometric interpretation of the weight optimization problem. The homotopy approach accounts for distinct sets of weights and infinite weights. The geometric interpretation further aids in explaining the appearance of local minima in the network, the appearance of infinite weights, and the similarities and differences between optimizing the weights in a nonlinear network, and the weights in a linear network.