Multiview face emotion recognition using geometrical and texture features
In the last decade, facial emotion recognition has attracted more and more interest of researchers in the computer vision community. Facial emotions are a form of nonverbal communication, used to exchange social and emotional information in human-human-interaction. By finding the emotion from...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2017
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/68532/1/FK%202018%2021%20-%20IR.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| id |
my-upm-ir.68532 |
|---|---|
| record_format |
uketd_dc |
| institution |
Universiti Putra Malaysia |
| collection |
PSAS Institutional Repository |
| language |
English |
| topic |
Human face recognition (Computer science) |
| spellingShingle |
Human face recognition (Computer science) Goodarzi, Farhad Multiview face emotion recognition using geometrical and texture features |
| description |
In the last decade, facial emotion recognition has attracted more and more
interest of researchers in the computer vision community. Facial emotions are a
form of nonverbal communication, used to exchange social and emotional
information in human-human-interaction. By finding the emotion from the
human face automatically and reacting proactively, several applications could
benefit. The examples of these are the human-computer-interfaces or security
systems, driver safety systems and social science's domain. In order to use
facial emotion recognition systems in real time situations, it is essential to
recognize emotions not only from frontal face images but also from images
containing faces with pose variations. Furthermore, facial landmarks have to be
located automatically. The degree of intensity of human facial emotions varies
from person to person. Some people may express the seven basic emotions more
intense than others or they may use it in different ways. In this thesis, a real
time emotion recognition system is presented. The system works on both,
frontal and non-frontal faces. A 3D face pose estimation algorithm detects head
rotations of Yaw, Roll and Pitch for emotion recognition. UPM3D-FE and
BU3D-FE databases are used in this research for training purposes which
include rotation and capturing of faces in different angles. After detecting the
human face, several features are extracted from human face automatically and
the geometrical facial features combined with texture features, are given to a
back propagation neural network which is trained with various face images.
This enables us to determine the emotion in real-time from the face of a person.
Basically, the contributions are that the method is capable of detecting the face
and facial landmarks in the live video; the landmark detection on the face is
done automatically in each frame using both texture of facial points and
relative positions of points on the face. Also, the emotion is detected from
frontal and angled face and in the case where half of face is not visible (side
view) the other half is reconstructed and emotion is detected. Geometrical and
texture features are used for emotion recognition and the texture features are taken from specific areas of the face in a novel approach. The results show an
improvement over existing approaches in determining emotions for various face
poses. The effects of gender, ethnicity, color, mixed emotions and intensity of
emotion have been analyzed as well. The resulting face emotion recognition
system works real time in less than twenty milliseconds per frame. For
UPM3DFE, in case of seven emotions, the accuracy is 63.08% for multiview
and 62.19% for near frontal faces for emotion recognition, and for the BU3DFE,
80.61% accuracy was found for near frontal faces and 77.48% for multi view in
the case of seven basic emotions. The achieved face emotion recognition
method has improved emotion recognition accuracy and also it is able to adapt
to the yaw and pitch rotation of face. Both databases (UPM3D and BU3D)
were tested for the role of gender, ethnicity, color, mixed emotions and intensity
of emotions. After cross validation, for the BU3DFE database, the best results
were achieved for Indians and Southeast Asian (56.6% and 50.2%) subjects. In
the case of UPM3DFE, the best results were achieved for Middle east and
southeast Asians subjects (66.6% and 69.1%), and the lowest results were
achieved in both databases for black subjects (45% and 54.54%). With regard
to mixed emotions, it has been found that BU3DFE is 67.72% accurate in
recognizing mixed emotions and UPM3DFE accuracy is 56.09%. In case of
different emotion intensities in BU3DFE, the results for multi view faces
manifested 71.11% for 1st emotion intensity, and 73.21% for 2nd emotion
intensity, 75.1% for 3rd emotion intensity and 79.31% for 4th emotion intensity. |
| format |
Thesis |
| qualification_level |
Doctorate |
| author |
Goodarzi, Farhad |
| author_facet |
Goodarzi, Farhad |
| author_sort |
Goodarzi, Farhad |
| title |
Multiview face emotion recognition using geometrical and texture features |
| title_short |
Multiview face emotion recognition using geometrical and texture features |
| title_full |
Multiview face emotion recognition using geometrical and texture features |
| title_fullStr |
Multiview face emotion recognition using geometrical and texture features |
| title_full_unstemmed |
Multiview face emotion recognition using geometrical and texture features |
| title_sort |
multiview face emotion recognition using geometrical and texture features |
| granting_institution |
Universiti Putra Malaysia |
| publishDate |
2017 |
| url |
http://psasir.upm.edu.my/id/eprint/68532/1/FK%202018%2021%20-%20IR.pdf |
| _version_ |
1747812593189257216 |
| spelling |
my-upm-ir.685322019-05-14T07:16:57Z Multiview face emotion recognition using geometrical and texture features 2017-10 Goodarzi, Farhad In the last decade, facial emotion recognition has attracted more and more interest of researchers in the computer vision community. Facial emotions are a form of nonverbal communication, used to exchange social and emotional information in human-human-interaction. By finding the emotion from the human face automatically and reacting proactively, several applications could benefit. The examples of these are the human-computer-interfaces or security systems, driver safety systems and social science's domain. In order to use facial emotion recognition systems in real time situations, it is essential to recognize emotions not only from frontal face images but also from images containing faces with pose variations. Furthermore, facial landmarks have to be located automatically. The degree of intensity of human facial emotions varies from person to person. Some people may express the seven basic emotions more intense than others or they may use it in different ways. In this thesis, a real time emotion recognition system is presented. The system works on both, frontal and non-frontal faces. A 3D face pose estimation algorithm detects head rotations of Yaw, Roll and Pitch for emotion recognition. UPM3D-FE and BU3D-FE databases are used in this research for training purposes which include rotation and capturing of faces in different angles. After detecting the human face, several features are extracted from human face automatically and the geometrical facial features combined with texture features, are given to a back propagation neural network which is trained with various face images. This enables us to determine the emotion in real-time from the face of a person. Basically, the contributions are that the method is capable of detecting the face and facial landmarks in the live video; the landmark detection on the face is done automatically in each frame using both texture of facial points and relative positions of points on the face. Also, the emotion is detected from frontal and angled face and in the case where half of face is not visible (side view) the other half is reconstructed and emotion is detected. Geometrical and texture features are used for emotion recognition and the texture features are taken from specific areas of the face in a novel approach. The results show an improvement over existing approaches in determining emotions for various face poses. The effects of gender, ethnicity, color, mixed emotions and intensity of emotion have been analyzed as well. The resulting face emotion recognition system works real time in less than twenty milliseconds per frame. For UPM3DFE, in case of seven emotions, the accuracy is 63.08% for multiview and 62.19% for near frontal faces for emotion recognition, and for the BU3DFE, 80.61% accuracy was found for near frontal faces and 77.48% for multi view in the case of seven basic emotions. The achieved face emotion recognition method has improved emotion recognition accuracy and also it is able to adapt to the yaw and pitch rotation of face. Both databases (UPM3D and BU3D) were tested for the role of gender, ethnicity, color, mixed emotions and intensity of emotions. After cross validation, for the BU3DFE database, the best results were achieved for Indians and Southeast Asian (56.6% and 50.2%) subjects. In the case of UPM3DFE, the best results were achieved for Middle east and southeast Asians subjects (66.6% and 69.1%), and the lowest results were achieved in both databases for black subjects (45% and 54.54%). With regard to mixed emotions, it has been found that BU3DFE is 67.72% accurate in recognizing mixed emotions and UPM3DFE accuracy is 56.09%. In case of different emotion intensities in BU3DFE, the results for multi view faces manifested 71.11% for 1st emotion intensity, and 73.21% for 2nd emotion intensity, 75.1% for 3rd emotion intensity and 79.31% for 4th emotion intensity. Human face recognition (Computer science) 2017-10 Thesis http://psasir.upm.edu.my/id/eprint/68532/ http://psasir.upm.edu.my/id/eprint/68532/1/FK%202018%2021%20-%20IR.pdf text en public doctoral Universiti Putra Malaysia Human face recognition (Computer science) |