Bayesian model averaging (BMA) is an approach to average over alternativemodels; yet, it usually gets excessively concentrated around the single mostprobable model, therefore achieving only sub-optimal classificationperformance. The compression-based approach (Boulle, 2007) overcomes thisproblem, averaging over the different models by applying a logarithmicsmoothing over the models' posterior probabilities. This approach has shownexcellent performances when applied to ensembles of naive Bayes classifiers.AODE is another ensemble of models with high performance (Webb, 2005), based ona collection of non-naive classifiers (called SPODE) whose probabilisticpredictions are aggregated by simple arithmetic mean. Aggregating the SPODEsvia BMA rather than by arithmetic mean deteriorates the performance; instead,we aggregate the SPODEs via the compression coefficients and we show that theresulting classifier obtains a slight but consistent improvement over AODE.However, an important issue in any Bayesian ensemble of models is thearbitrariness in the choice of the prior over the models. We address thisproblem by the paradigm of credal classification, namely by substituting theunique prior with a set of priors. Credal classifier automatically recognizethe prior-dependent instances, namely the instances whose most probable classvaries, when different priors are considered; in these cases, credalclassifiers remain reliable by returning a set of classes rather than a singleclass. We thus develop the credal version of both the BMA-based and thecompression-based ensemble of SPODEs, substituting the single prior over themodels by a set of priors. Experiments show that both credal classifiersprovide higher classification reliability than their determinate counterparts;moreover the compression-based credal classifier compares favorably to previouscredal classifiers.
展开▼
