Universal Knowledge-Seeking Agents for Stochastic Environments

摘要

We define an optimal Bayesian knowledge-seeking agent, KL-KSA, designed for countable hypothesis classes of stochastic environments and whose goal is to gather asmuch information about the unknown world as possible.Although this agentworks for arbitrary countable classes and priors, we focus on the especially interesting case where all stochastic computable environments are considered and the prior is based on Solomonoff's universal prior. Among other properties, we show that KLKSA learns the true environment in the sense that it learns to predict the consequences of actions it does not take.We show that it does not consider noise to be information and avoids taking actions leading to inescapable traps.We also present a variety of toy experiments demonstrating thatKLKSA behaves according to expectation.