Generalized Gaussian process models with Bayesian variable selection.

摘要

This research proposes a unified Gaussian process modeling approach that extends to data from the exponential dispersion family and survival data. Our specific interest is in the analysis of datasets with predictors possessing an a priori unknown form of possibly non-linear associations to the response. We incorporate Gaussian processes in a generalized linear model framework to allow a flexible non-parametric response surface function of the predictors. We term these novel classes "generalized Gaussian process models". We consider continuous, categorical and count responses and extend to survival outcomes. Next, we focus on the problem of selecting variables from a set of possible predictors and construct a general framework that employs mixture priors and a Metropolis-Hastings sampling scheme for the selection of the predictors with joint posterior exploration of the model and associated parameter spaces.;We build upon this framework by first enumerating a scheme to improve efficiency of posterior sampling. In particular, we compare the computational performance of the Metropolis-Hastings sampling scheme with a newer Metropolis-within-Gibbs algorithm. The new construction achieves a substantial improvement in computational efficiency while simultaneously reducing false positives. Next, leverage this efficient scheme to investigate selection methods for addressing more complex response surfaces, particularly under a high dimensional covariate space.;Finally, we employ spiked Dirichlet process (DP) prior constructions over set partitions containing covariates. Our approach results in a nonparametric treatment of the distribution of the covariance parameters of the GP covariance matrix that in turn induces a clustering of the covariates. We evaluate two prior constructions: The first employs a mixture of a point-mass and a continuous distribution as the centering distribution for the DP prior, therefore clustering all covariates. The second one employs a mixture of a spike and a DP prior with a continuous distribution as the centering distribution, which induces clustering of the selected covariates only. DP models borrow information across covariates through model-based clustering, achieving sharper variable selection and prediction than what obtained using mixture priors alone. We demonstrate that the former prior construction favors "sparsity", while the latter is computationally more efficient.