Dynamical Stability Conditions for Recurrent Neural Networks with Unsaturating Piecewise Linear Transfer Functions

摘要

We establish two conditions that ensure the nondivergence of additive re- current networks with unsaturating piecewise linear transfer functions, also called linear threshold or semilinear transfer functions. As Hahn- loser, Sarpeshkar, Mahowald, Douglas, and Seung (2000) showed, net- works of this type can be efficiently built in silicon and exhibit the co- existence of digital selection and analog amplification in a single circuit. To obtain this behavior, the network must be multistable and nondiver- Gent, and our conditions allow determining the regimes where this can be Achieved with maximal recurrent amplification. The first condition can be Applied to nonsymmetric networks and has a simple interpretation of re- Quiring that the strength of local inhibition match the sum over excitatory Weights converging onto a neuron.