Optimization in Non-parametric Survival Analysis and Climate Change Modeling.

摘要

Many of the open problems of current interest in probability and statistics involve complicated data sets that do not satisfy the strong assumptions of being independent and identically distributed. Often, the samples are known only empirically, and making assumptions about underlying parametric distributions is not warranted by the insufficient information available. Under such circumstances, the usual Fisher or parametric Bayes approaches cannot be used to model the data or make predictions.;However, this situation is quite often encountered in some of the main challenges facing statistical, data-driven studies of climate change, clinical studies, or financial markets, to name a few.;We propose a novel approach, based on large deviations theory, convex optimization, and recent results on surrogate loss functions for classifier-type problems, that can be used in order to estimate the probability of large deviations for complicated data. This may include, for instance, highdimensional data, highly-correlated data, or very sparse data.;The thesis introduces the new approach, reviews the current known theoretical results, and then presents a number of numerical explorations meant to quantify how far the approximation of survival functions via large deviations principle can be taken, once we leave the limitations imposed by the existing theoretical results.;The explorations are encouraging, indicating that indeed the new approximation scheme may be very efficient and can be used under much more general conditions than those warranted by the current theoretical thresholds.;After applying the new methodology to two important contemporary problems (atmospheric CO2 data and El Niño/La Niña phenomena), we conclude with a summary outline of possible further research.