Auditory attention tracking states in a cocktail party environment can be decoded by deep convolutional neural networks

摘要

Objective. A deep convolutional neural network (CNN) is a method for deep learning (DL). It has apowerful ability to automatically extract features and is widely used in classification tasks withscalp electroencephalogram (EEG) signals. However, the small number of samples and lowsignal-to-noise ratio involved in scalp EEG with low spatial resolution constitute a limitation thatmight restrict potential brain-computer interface (BCI) applications that are based on the CNNmodel. In the present study, a novel CNN model with source-spatial feature images (SSFIs) as theinput is proposed to decode auditory attention tracking states in a cocktail party environment.Approach. We first extract SSFIs using rhythm entropy and weighted minimum norm estimation.Next, we develop a CNN model with three convolutional layers. Furthermore, we estimate theperformance of the proposed model via generalized performance, alternative models that deletedor replaced a model’s component, and loss curves. Finally, we use a deep transfer model withfine-tuning for a low (poor) behavioral performance group (L-group). Main results. Based oncortical activity reconstructions from the scalp EEGs, the classification accuracy (CA) of theproposed model is 80.4% (chance level: 52.5%), which is superior to that achieved by scalp EEG.Additionally, the performance of the proposed model is more stable when compared to alternativemodels that delete or replace specific model components. The proposed model identifies thedifference between two auditory attention tracking states (successful versus unsuccessful) at anearly stage with a short time window (250 ms after target offset). Furthermore, we propose a deeptransfer learning model to improve the classification for the L-group. With this model, the CA ofthe L-group significantly increase by 5.3%. Significance. Our proposed model improves theperformance of a decoder for auditory attention tracking, which could be suitable for relieving thedifficulty with the attentional modulation of individual’s neural responses. It provides a novelcommunication channel with auditory cognitive BCI for patients with attention and hearingimpairment.