Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

摘要

Simple probabilistic reinforcement learning is recognized asa striatum-based learning system, but in recent years, has alsobeen associated with hippocampal involvement. This studyexamined whether such involvement may be attributed toobservation-based learning (OL) processes, running in parallelto striatum-based reinforcement learning. A computationalmodel of OL, mirroring classic models of reinforcement-basedlearning (RL), was constructed and applied to the neuroimagingdata set of Palombo, Hayes, Reid, and Verfaellie 2019.Hippocampal contributions to value-based learning: Convergingevidence from fMRI and amnesia. Cognitive, Affective BehavioralNeuroscience, 19(3), 523–536. Results suggested that OLprocesses may indeed take place concomitantly to reinforcementlearning and involve activation of the hippocampus andcentral orbitofrontal cortex. However, rather than independentmechanisms running in parallel, the brain correlates of the OLand RL prediction errors indicated collaboration between systems,with direct implication of the hippocampus in computationsof the discrepancy between the expected and actualreinforcing values of actions. These findings are consistent withprevious accounts of a role for the hippocampus in encoding thestrength of observed stimulus–outcome associations, withupdating of such associations through striatal reinforcementbasedcomputations. In addition, enhanced negative RL predictionerror signaling was found in the anterior insula with greateruse of OL over RL processes. This result may suggest an additionalmode of collaboration between the OL and RL systems,implicating the error monitoring network.