In practice, analog neural networks appear to be more useful than binary; however, until now their behavior was very difficult to analyze. We propose k-ary neural networks (a discrete multivalued model) as a tool for reasoning about aspects of the behavior of limited-precision analog neural networks.;First, we show several computational results for k-ary neural networks and related multithreshold models. For example: (1) For polynomially bounded size and weights and polylogarithmic precision, two hidden layers are more powerful than a single hidden layer. (2) There is no canonical set of threshold values for k ;Second, we discuss learning with k-ary neural networks. In particular, we present: (1) The generalization of the binary perceptron algorithm to a k-ary learning algorithm with a guaranteed convergence property. (2) An extension of the Littlestone's binary Winnow algorithm to an exceptionally fast k-ary learning algorithm with minimal architecture requirements.
展开▼
