Learning Systems for Minimum Risk Adaptive Pattern Classification and Optimal Adaptive Estimation

摘要

The two problems of unsupervised learning, sequential multi-category pattern classification and adaptive estimation of a sampled stochastic process are jointly investigated. An unknown parameter model is developed which, for the pattern classification problem, allows for (i) both constant and time-varying unknown parameters, (ii) partially unknown probability laws of the hypotheses and time-varying parameter sequences, (iii) dependence of the observations on past as well as present hypotheses and parameters, and most significantly (iv) sequential dependencies in the observations arising from either (or both) dependency in the pattern or information source (context independence) or in the observation medium (measurement correlation), these dependencies being up to any finite Markov orders. For the adaptive estimation problem the same model is employed without any distinction between 'hypotheses' and 'time-varying parameters.' For finite parameter spaces, the solutions which are Bayes optimal (minimum risk) at each step are found for both problems and shown to be realizable in recursive form with fixed memory requirements. The recursive 'learning' portion of the solutions is the same for both problems. The asymptotic properties of the optimal systems are studied and conditions established for these systems (in addition to making best use of available data at each step) to converge in performance to systems operating with knowledge of the (unobservable) constant unknown parameters. (Author)