Gradient and Hamiltonian dynamics: Some applications to neural network analysis and system identification.

摘要

The work in this dissertation is based on decomposing system dynamics into the sum of dissipative (e.g. convergent) and conservative (e.g. periodic) components. Intuitively, this can be viewed as decomposing the dynamics into a component normal to some surface and components tangent to other surfaces. First, this decomposition was applied to existing neural network architectures to analyze their dynamic behavior. Second, this formalism was employed to create models which learn to emulate the behavior of actual systems. The premise of this approach is that the process of system identification can be considered in two stages: model selection and parameter estimation. In this dissertation a technique is presented for constructing dynamical systems with desired qualitative properties. Thus, the model selection stage consists of choosing the dissipative and conservative portions appropriately so that a certain behavior is obtainable. By choosing the parametrization of the models properly, a learning algorithm has been devised and proven to always converges to a set of parameters for which the error between the output of the actual system and the model vanishes. So these models and the associated learning algorithm are guaranteed to solve certain types of nonlinear identification problems.