Enhanced Equivalence Projective Simulation: A Framework forModeling Formation of Stimulus Equivalence Classes

摘要

Formation of stimulus equivalence classes has been recently modeledthrough equivalence projective simulation (EPS), a modified versionof a projective simulation (PS) learning agent. PS is endowed with anepisodic memory that resembles the internal representation in the brainand the concept of cognitive maps. PS flexibility and interpretabilityenable the EPS model and, consequently the model we explore in thisletter, to simulate a broad range of behaviors in matching-to-sample experiments.The episodic memory, the basis for agent decision making, isformed during the training phase. Derived relations in the EPS modelthat are not trained directly but can be established via the network’s connectionsare computed on demand during the test phase trials by likelihoodreasoning. In this letter, we investigate the formation of derivedrelations in the EPS model using network enhancement (NE), an iterativediffusion process, that yields an offline approach to the agent decision making at the testing phase. The NE process is applied after the trainingphase to denoise the memory network so that derived relations areformed in the memory network and retrieved during the testing phase.During the NE phase, indirect relations are enhanced, and the structureof episodic memory changes. This approach can also be interpreted as theagent’s replay after the training phase, which is in line with recent findingsin behavioral and neuroscience studies. In comparisonwith EPS, ourmodel is able to model the formation of derived relations and other featuressuch as the nodal effect in a more intrinsic manner. Decision makingin the test phase is not an ad hoc computational method, but rathera retrieval and update process of the cached relations from the memorynetwork based on the test trial. In order to study the role of parameterson agent performance, the proposed model is simulated and the resultsdiscussed through various experimental settings.