Computation of the Frechet mean, variance and interpolation for a pool of neural networks over the manifold of special orthogonal matrices

摘要

The present manuscript tackles the problem of merging the connection patterns learnt by a pool of neural networks that share the manifold of special orthogonal matrices as parameter space. The merging technique is implemented as an averaging algorithm over the curved manifold of special orthogonal matrices. In the present manuscript, averaging is computed via the notion of Frechet mean and the associated metric dispersion is interpreted as the variance of the patterns around the Frechet mean. Also, continuous interpolation of two connection patterns is considered as an extension of the Frechet principle.