Preferential training of neurodynamical model based on predictability of target dynamics

摘要

Intrinsic motivation is one of the keys in implementing the mechanism of interest to robots. In this paper, we present a method to apply intrinsic motivation in dynamics learning with predictable and unpredictable targets in view. The robot's arm is used for the predictable target and the human's arm is used for the unpredictable target in the experiment. The learning algorithm based on intrinsic motivation will automatically set a larger weight to targets that would contribute to decreasing the training error, while setting a smaller weight to others. A neurodynamical model, namely multiple timescales recurrent neural network (MTRNN), is utilized for studying the robot's arm/external object dynamics. Training of MTRNN is done using the back propagation through time (BPTT) algorithm. We modify the BPTT algorithm by the following two steps. (1) Evaluate predictability of robot's arm/objects using training error of MTRNN. (2) Assign a preference ratio, which represents the weight of the training, to each object based on predictability. The proposed training method would focus more on reducing training error of predictable objects compared to normal BPTT, where training error is equally treated for every object. Experiments were conducted using an actual robot platform, moving the robot's arm while a human moves his arm in the robot's camera view. The results of the experiment showed that the proposed training method could achieve smaller training error of the robot's arm visuomotor dynamics, which is predictable from the robot's motor command, compared to general training with BPTT. Evaluation of MTRNN as a forward model to predict untrained data showed that the proposed model is capable of predicting the robot's hand motion, specifically with larger number of nodes.