Neural Learning and Weight Flow on Stiefel Manifold

摘要

The aim of this paper is to present a new class of learning models for linear as well as non-linear neural layers called Orthonormal Strongly-Constrained (SOC or Stiefel). They allow to solve orthonormal problems where orthonormal matrices are involved. After general properties of the learning rules belonging to this new class are shown, examples derived independently or by reviewing learning theories known from the literature are presented and discussed.