CNN HMI HMI CONVOLUTIONAL NEURAL NETWORK BASED HUMAN MACHINE INTERFACE SYSTEM USING DOPPLER RADAR AND VOICE SENSOR DEVICE FOR PROCESSING SENSOR DATA OF THE HUMAN MACHINE INTERFACE SYSTEM METHOD FOR OPERATING THE SAMES

摘要

The present invention relates to a CNN-based HMI system using a Doppler radar and a voice sensor, a sensor data processing apparatus of the HMI system, and an operation method thereof. It aims to improve classification performance by fusion of sensor information. In one example, the HMI sensor unit senses the user''s voice command and the gesture command, respectively, and outputs the voice signal and the gesture signal, respectively; a Fourier transform unit that converts the voice signal and the gesture signal into a frequency signal for time change through Fourier transform, respectively, and outputs the converted frequency signal as spectrogram data; and extracting valid feature data through a convolution operation on voice spectrogram data and gesture spectrogram data respectively output through the Fourier transform unit, and extracting the extracted feature data into one through a filter operation of a Fully-Connected Layer (FCL). Disclosed is a CNN-based HMI system including a CNN model unit classified into a class of