Image Reconstruction from Neural Activity Recorded from Monkey Inferior Temporal Cortex Using Generative Adversarial Networks

摘要

The generative adversarial network (GAN) is a powerful image generation machine learning model. Several lines of research have shown that GAN is applicable to brain-machine interface technology for deciphering human brain activity, such as EEG and fMRI signals, to visualize what human observers see during recording. However, although current GAN models can synthesize photorealistic images, the quality and variety of image reconstruction from brain activity data recorded by non-invasive techniques are still limited. In this study, we recorded neural spike activities in monkey brain using microelectrode arrays implanted directly on the surface of the inferior temporal cortex, a brain area crucial for visual object recognition. The recorded data were then inputted into a state-of-the-art GAN model (Dosovitskiy & Brox, 2016 [1]) to reconstruct images viewed by the monkey during the experiments. The results showed the advantage of invasive recording methods over non-invasive methods for improving the quality of image reconstruction. The results also demonstrated that the proposed decoding approach is useful in neuroscience research to explore and visualize information represented in the recoding site.