An analysis of the metric structure of the weight space of feedforward networks and its application to time series modeling and prediction

摘要

We study symmetries of feedforward networks in terms of their corresponding groups. We find that these groups naturally act on and partition weight space into disjunct domains. We derive an algorithm to generate representative weight vectors in a fundamental domain. The analysis of the metric structure of the fundamental domain leads to improved evaluation procedures of learning results, such as local error bars estimated using maximum-likelihood and bootstrap methods. It can be implemented effciently even for large networks. We demonstrate the approach in th4e area of nonlinear time series modeling and prediction.