Biases in estimating the balance between model-free and model-based learning systems due to model misspecification

摘要

Accumulating studies support the existence of dual learning systems in decision making: one is the model-free learning system, which updates values based solely on experience; and the other is the model-based learning system, which calculates values using a complex environmental structure. A two stage decision task and its computational model have been widely used to distinguish the effects of these systems on choices. However, the computational model is often used as a tool without doubting its assumptions. In this study, we examined the possible biases in model parameter estimation due to model misspecification of a computational model. In particular, we focused on two features related to choice behavior, the existence of which was implied by the actual choice data but has not been assumed in the widely used computational models. One feature is the forgetting process, which assumes a change in unchosen option values. The other feature is gradual perseveration, which assumes that actions are positively autocorrelated with multiple preceding actions. We simulated cases in which these features relate to the choice process, but the obtained data were fit using a model that does not assume these features. We revealed that such misspecification of a fitting model can cause systematic biases in the estimation of the relative contributions of the model-free and model-based systems, implying that previous findings using the standard computational model might have been distorted by some biases. The possibility of estimation biases discussed in this study is important because the assumptions of the forgetting process and gradual perseveration, which can be combined with any reinforcement learning model, are not included in most existing models. In addition, the discussed mechanisms of the biases are widely related to basic model parameters. Using experimental data from the two-stage decision task (N = 39), we examined the associations between obsessive compulsivity and the w