Bayesian design of experiments for logistic regression to evaluate multiple nuclear forensic algorithms

摘要

When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. We develop methodology for the case where there are several potentially nonoverlapping priors for anticipated performance under consideration. The weighted priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other candidate design choices. Additionally, we show how this can be applied in the multivariate input case and provide some useful summary measures. The shared information plot is used to evaluate design point allocation, and the D-value difference plot allows for the comparison of design performance across multiple potential parameter values in higher dimensions. We illustrate the methods with several examples.