EEG HUMAN EMOTION STATE RECOGNITION METHOD AND SYSTEM USING FUSION OF IMAGE AND EEG SIGNALS

摘要

Disclosed is a method and system for recognizing a human emotional state using image and EEG signal fusion. The emotion recognition method performed by the emotion recognition system according to an embodiment is to construct a convergence network that fuses feature information extracted from the video signal and the electrical biometric signal through a modality network for each of the video signal and the electrical biometric signal. step; Each weight for the balance data output through the modality network for each of the video signal and the electrical bio-signal based on each reliability determined using the feature information of the video signal and the electrical bio-signal in the constructed converged network. Determining; And outputting a final balance value as output data by reflecting each weight determined for the output balance data, wherein the emotion recognition method includes: a modality network for the video signal, and an electrical biosignal for the signal. The modality network and the convergence network may be separately learned.