METHOD OF PROCESSING VIDEO FOR DETERMINING EMOTION OF A PERSON

摘要

The multi-modal emotion recognition method using artificial intelligence of the present description is an emotion recognition method that processes an image to determine a person's emotional state, and provides an image and a voice expressing the appearance of a person, wherein the image is the first comprising an imaging unit, a second imaging unit immediately following the first imaging unit, and a third imaging unit immediately following the second imaging unit; The first image unit processes the first image unit to determine the emotional state of the person, wherein the first image unit shows the person's face and at least one hand, and the at least one hand characterized in that no part of the face overlaps; and the second image unit processes the second image unit to determine the emotional state of the person, wherein the second image unit shows the person's face and at least one hand, and the at least one hand is the person Including, wherein the processing of the first image unit includes at least one of the first image unit to determine whether the at least one hand covers the face of the person processing a frame of ; finding a first facial element of the person in the at least one frame of the first image unit; and in a state in which the first facial element is located, the at least one obtaining first facial feature data of the first image unit based on the shape of the first facial element shown in the frame of processing the audio data of the first image unit to obtain determining a state; wherein the processing of the second image unit includes processing at least one frame of the second image unit to determine whether the face of the person is covered by at least one hand. determining whether or not the at least one hand covers the face of the person in the second image unit; and finding the first facial element of the person in at least one frame of the second image unit. obtaining the first facial feature data of the first image unit based on the shape of the first face element shown in the at least one frame of the second image unit in a state where the first facial element is located step, and the audio of the second image unit to obtain the voice characteristic data based on the human voice characteristics in the second image unit processing data, the first facial feature data of the second image unit, the audio feature data of the second image unit, and additional data indicating a location where at least one hand covers a part of the person's face and determining the emotional state of the person for the second image unit based on the plurality of data included therein.