Facial Expression Recognition using Facial Landmark Detection and Feature Extraction on Neural Networks

摘要

The proposed framework in this paper has the primary objective of classifying the facial expression shown by a person using facial landmark detection and feature extraction. These classifiable emotions can be any one of the six universal emotions: happiness, sadness, anger, disgust, surprise and fear. After initial image pre-processing to reduce image noise, facial detection is performed on the image after which the RoI is extracted, upon which facial landmark detection and facial feature extraction is performed (where in the facial landmarks were determined to be the fiducial eyebrows, eyes, nose and mouth), which is done using the Sobel horizontal edge detection method and the Shi Tomasi corner point detector. This finally leads to input feature vectors being formulated and trained into a feed forward back propagation Multi Layer Perceptron (MLP) Neural Network in order to classify the emotion being displayed. Facial Expression Recognition (FER) is a significant step in reaching the eventual goal of artificial intelligence. If efficient methods can be brought about to automatically recognize these facial expressions, striking improvements can be achieved in the area of human computer interaction. AI has long relied on the area of facial emotion recognition to gain intelligence on how to model human emotions convincingly in robots. An application such as Siri could be largely improved to detect its users emotion and thus, better serve its purpose by giving more appropriate results accordingly.