Zebra finch vocal development through reinforcement of the anterior forebrain pathway.

摘要

This dissertation explores reinforcement learning in the context of zebra finch song development. In the first chapter, we explore why we believe reinforcement learning is present in songbirds. We argue that similarities in vocal learning and neurophysiology between humans and songbirds offer compelling evidence that the two develop vocalization under similar constraints. A general constraint of motor learning tasks in animals is the requirement of the neurotransmitter dopamine. In mammals dopamine has been shown to encode reward information that is subsequently used to reinforce the motor activity.;The second chapter presents our reinforcement learning model in the context of song development during the sensorimotor phase of zebra finch. We use simplified binary neurons and synaptic plasticity rules to model activity in critical nuclei that are involved in zebra finch song learning. The model generates exploration in the anterior forebrain pathway (AFP) to guide the song trajectory to a stored tutor song. More specifically, random activity in the lateral magnocellular nucleus of the anterior nidopallium (LMAN) drives random exploration of HVC (proper name) projections to area X. When the model's juvenile song moves towards the memorized tutor template a reward is generated. The reward is represented by activity in the ventral tegmental area (VTA) which globally projects dopamine to area X. The reinforcement of area X activity is permanently mapped onto the premotor projection from HVC to the robust nucleus of the arcopallium (RA). The reward activity is delayed by 100 ms for biological reasons creating a temporal difference problem between activity and its corresponding reward. We resolve this issue using sustained area X activity and a plastic excitatory projection from area X to the VTA. The model is able to guide song development to the tutor song template by using biologically reasonable connectivity and synaptic learning rules.;Following the presentation of our birdsong model, a brief summary of a computational neuron model previously developed is presented in chapter 3. We look at the important aspects of neurons contained in the medial nucleus of the dorsolateral thalamus (DLM). The DLM neurons show unique activity transmission within the AFP and its role in song learning is unknown. The computational model is a single compartment, conductance based neuron with several ion channels. The properties of the ion channels were derived from a neuron model of mammalian thalamic relay neurons. The model reproduces the electrophysiological properties experimentally reported for the DLM neuron that are critical to the timing reported in the AFP.