Bayesian classification using Bayesian additive and regression trees.

摘要

In this dissertation, we developed a Bayesian approach to classification problems. Classification problems range from recognizing handwritten text to predicting whether an individual will develop a certain disease. To address classification problems, various methods (for example, Classification Trees, Artificial Neural Networks, Support Vector machines) have been developed. However, still there are problems where classification error rate is 10% or higher; in addition, applications of classifiers have shown that estimating class membership probabilities, and their uncertainty, is very important aspect of classification problems. In this dissertation, we addressed the above issues by developing a new classifier: CBART. The classifier is based on Bayesian Additive and Regression trees (BART); we used latent variables to extend BART to binary and multiclass ordered classification problems. Our investigation has shown that CBART provides error rates and AUC (Area Under the Curve) comparable with those of benchmark classifiers. The benefits of CBART are that it provides class membership probabilities and their distributions; hence, using CBART, we have a measure of the uncertainty related to class membership probability. Other methods such as Logistic Regression (LR) provide class membership probabilities and their distribution. LR, however, assumes a linear model. Instead, CBART, being based on classification trees, is able to deal with non-continuous classification problems. Furthermore by using a Tree-based approach, CBART automatically selects among the available predictors. Other classifiers (e.g. Logistic Regression and Neural Networks) use all available predictors. Finally, in this research we provided methods that can be used to measure classifiers' importance, and we have provided ideas for future research.