Transfer Learning-Based Behavioural Task Decoding from Brain Activity

摘要

Brain decoding bears a high potential for future applications in medical sciences and healthcare industries. It can predict individual brain differences and diagnose from neuroimaging data, offering new paths for treatment and prevention. Recent advances in functional magnetic resonance imaging (fMRI) techniques have shown that it is possible to decode visual information from measurable brain activities. However, these studies typically focus on the decoding of image and video stimulus within the subject. To overcome these limitations, we proposed a cross-subject decoding approach based on deep transfer learning to decode the brain state from task-based fMRI. In detail, we designed a pipeline for brain decoding. One can use parts of the state-of-the-art networks pre-trained on ImageNet data set connect to our defined layers to complete the target decoding tasks. The experiments in this study are implemented on the Human Connectome Project (HCP) data set. The results show that we obtained a higher accuracy of cross-subject decoding compared to previous academic studies. We further demonstrate that fully trained convolution neural network (CNN) and pre-trained CNN with fine-tuning outperformed existing methods on the same database.