Real-time Malaysian Sign Language recognition system using Microsoft Kinect 360 based on locally linear embedding and artificial neural network model /
Deaf people or people with hearing loss have a major problem in everyday communication. Sign Language (SL) is a common communication method for deaf people. Many attempts have been made with SL translator to solve of communication gap between normal and deaf people and ease communication for deaf pe...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
Kuala Lumpur :
Kulliyyah of Information and Communication Technology, Internatrional Islamic University Malaysia,
2017
|
| Subjects: | |
| Online Access: | Click here to view 1st 24 pages of the thesis. Members can view fulltext at the specified PCs in the library. |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Deaf people or people with hearing loss have a major problem in everyday communication. Sign Language (SL) is a common communication method for deaf people. Many attempts have been made with SL translator to solve of communication gap between normal and deaf people and ease communication for deaf people. The system is able to match and compare the input sign trajectory with each of the prototype sign trajectory contained in the database with lower error rate. This is achieved by extracting a number of static and dynamic features from right hand and left hand. This contribution tries to introduce an SL translator, especially for static and dynamic MSL by using Kinect 360 technology and Native signers with MSL database which have been created in this research. Iterative method has been used for data denoising for depth information. The result for denoising data has been reduce from 307200 to 160000 value. HOG and GA are used as feature extraction for static sign recognition. SVM classifier is used for training and testing the developed system using static signs. Accuracy result for static signs using HOG is 99.37%, GA is 62.92% and GA+HOG is 93.14%. LLE and PCA feature extraction has been used for dynamic sign recognition which improved accuracy result much better (it is mentioned that LLE features have been used for the first time for dynamic sign recognition). Three types of classifier such as MLP, CFNN and SVM are used to test and implement dynamic sign recognition. Accuracy results are 92.30%, 88.50% and 82.70% for MLP, CFNN and SVM respectively. The developed MSL recognition system was tested using 10 dynamic words and 24 static alphabets. The developed MSL recognition system has attained a significant performance in terms of recognition accuracy and speed that allow a real time translation of signs into text. |
|---|---|
| Physical Description: | xvii, 194 leaves : illustrations ; 30cm. |
| Bibliography: | Includes bibliographical references (leaves 177-188). |