Adaptive classifier design using labelled and unlabelled data.

摘要

In statistical pattern recognition the goal is to accurately classify samples based on a set of descriptive predictor variables (features). The design of pattern recognition systems involves three major tasks: acquisition of training data; data pre-processing; and learning a classification function that accurately predicts the class label of new samples. This thesis develops statistical algorithms for all three tasks as explained below.; First, consider pre-processing: in this stage of classifier design, the objective is to represent the data in a form that allows the classification function to be learnt accurately in the subsequent stage. Since many of the initially postulated predictor variables may be irrelevant, in order to be able to learn an accurate classification from limited training data, it is important to identify the predictor variables which carry information pertinent to the classification. Often referred to as feature selection, previous work has largely addressed this task in isolation from that of classifier design.; The first part of this thesis develops algorithms that combine these two tasks and solve them jointly. Removing the artificial distinction between these tasks, the algorithms presented here identify only those features that are most useful in performing the classification itself. For the special case of binary linear classifiers, non-trivial theoretical error bounds are derived for our algorithm. These bounds are significantly tighter than previous results such as those derived from Vapnik-Chervonenkis theory.; The second part of this thesis develops a new semi-supervised algorithm for learning the classification function. Conventional supervised classifier learning methods assume access to training data associated with known class labels. However, in many problems data is initially gathered unlabelled and the subsequent acquisition of labelled information is costly and/or time consuming. Given access to a small number of labelled samples and an abundance of unlabelled data, semi-supervised algorithms that learn from both have been the focus of much research in the last few years.; The last part of the thesis develops methods for adaptive collection of training data in a way that focuses on learning the classification function accurately, under the constraints of a limited data-acquisition budget. As compared to passively learning from examples provided by a teacher, a student can learn a concept much faster by actively asking questions to the teacher to clarify doubts. Based on this idea, active data query selection is performed using a mutual information based criterion that explicitly uses the labelled and unlabelled data as well as the co-training information to decide what additional data will be most useful for learning the classifier. (Abstract shortened by UMI.)