Information Theoretic Approach to Distributed Inference and Learning

摘要

This report describes research work which was funded under grant number AFOSR90-00199 during the period February 1st 1990 to May 31st 1992. Our work has focused on developing information-theoretic and probabilistic models for neural network computation. This theoretical basis is then used to develop novel hybrid network architectures, which combine techniques from the fields of statistics and artificial intelligence with neural approaches. The report describes a number of significant results including identification of the general class of energy functions which lead to proper probability estimation, a new algorithm which builds hybrid rule-based network models from data, Markov random field theory and algorithms for constructing network models from large databases, new results on sparse Markov models, a new hybrid unsupervised/supervised learning algorithm with applications to computer vision problems, a novel recurrent network structure, and prototype VLSI hardware implementations of these ideas. A Total of 30 technical papers have resulted from this grant.