Incomplete learning from endogenous data in dynamic allocation

摘要

This paper studies the problem of learning from endogenous data by an economic agent who chooses actions sequentially from a finite set {a_1,...,a_k} such that the reward R(a_j) of action a_j has a probability distribution depending on an unknown parameter θ_j that has a prior distribution II~((j)). The agent's objective is to maximize the total discounted reward ∫…∫E_(θ_1…θ_k){Σ from t = 0 to ∞ of β~tR(X_(t+1))}dII~((1))(θ_1)…dII~((k))(θ_k), where 0 < β < 1 is a discount factor and X_t denotes the action chosen by the agent at time t. The optimal solution to this problem, commonly called the "discounted multiarmed bandit problem," was shown by Gittins and Jones (1974) and Gittins (1979) to be the "index rule" that chooses at each stage the action with the largest "dynamic allocation index" (DAI). The theory of multi-armed bandits has been applied to decision making in labor markets (cf. Jovanovic (1979), Mortensen (1985)), general search problems involving nondurable goods (cf. Banks and Sundaram (1992)) and pricing under demand uncertainty (cf. Rothschild (1974)).