Representations of Highly-Varying Functions by One-Hidden-Layer Networks

摘要

Limitations of capabilities of one-hidden-layer networks are investigated. It is shown that for networks with Heaviside perceptrons as well as for networks with kernel units used in SVM, there exist large sets of d-variable functions which cannot be tractably represented by these networks, i.e., their representations require numbers of units or sizes of weighs depending on d exponentially. Our results are derived using the concept of variational norm from nonlinear approximation theory and the concentration of measure property of high dimensional Euclidean spaces.