A nonstationary environment is one in which the suitability of the strategies available to a learning element changes with time. Since the optimal action in such a case is not fixed, the learning problem (i.e., the determination of the optimal strategy) becomes considerably difficult. In this paper, a two-level approach is presented for a learning automaton operating in a nonstationary environment. The lower level consists of a standard absolutely expedient learning algorithm for stationary environments. The higher level on the other hand is a tracking algorithm, based on Bayesian decision theory, for detecting changes in the environemnt and reinitializing the lower level algorithm in a suitable manner. Simulation studies empirically demonstrate the clear superiority of the two-level approach over the single-level learning in nonstationary environments.
展开▼
