We consider the problem of sequence prediction in a probabilistic setting. Let there be given a class C of stochastic processes (probability measures on the set of one-way infinite sequences). We are interested in the question of what are the conditions on C under which there exists a predictor (also a stochastic process) for which the predicted probabilities converge to the correct ones if any of the processes in C is chosen to generate the data. We find some sufficient conditions on C under which such a predictor exists. Some of the conditions are asymptotic in nature, while others are based on the local (truncated to first observations) behaviour of the processes. The conditions lead to constructions of the predictors. In some cases we obtain rates of convergence that are optimal up to an additive logarithmic term. We emphasize that the framework is completely general: the stochastic processes considered are not required to be i.i.d., stationary, or to belong to some parametric family.
展开▼
