Reward-based learning and basal ganglia: A biologically realistic, computationally explicit theory.

摘要

Inspired by the suggestive complexity and specificity of the neurobiology of basal ganglia, as well as by its believed dominance of reward based behavior, the present work develops a computationally explicit and biologically concerned theory of the structure.; The basal ganglia receives input from almost the entire cortical mantle, while its output influences the activity of brainstem and frontal cortex. This connectivity reflects the ability of the basal ganglia to associate sensory context to behavior. The model considers that behavior is composed of the sum of the effects of concurrent actions with varying strengths of dominance. In this view, the output of the model has a continuous nature, an uncommon feature in reward based learning schemes, yet necessary to approach real world situations.; The basal ganglia also receives dopamine, which has been strongly implicated with signaling reward in animals. The dopamine signal allows the structure to support plasticity that associates sensory context with proper action based on reward. Given that reward is always delayed, the proposed model uses the chemical container properties of synaptic spines to handle continuous time delays.; The proposed model is tested in simulated and real world tasks. These tasks are designed to exercise the dominant type of learning believed to be implemented by basal ganglia, reward based learning. The experimental results demonstrate that the model developed is able to learn appropriately.; It is concluded that the proposed model is computationally sound, while capturing the main features and function of basal ganglia. To our knowledge, this is the first effort that produces a biologically realistic model of basal ganglia that supports continuous sensory, action, and time spaces.