Hand gesture classification using convex hull skeleton joints moment knn technique for dimensionality reduction and incremental learning in the central nervous system’s hologram
Recent breakthroughs with numerous visual experiences using mobile devices encourage the research of humancomputer interaction (HCI) involving hand gesture recognition for Holograms, Virtual Reality, and Augmented Reality. The rise of these technologies allows educators in medical segments to ap...
Saved in:
Main Author:  

Format:  Thesis 
Language:  English 
Published: 
2022

Subjects:  
Online Access:  http://psasir.upm.edu.my/id/eprint/104004/1/FSKTM%202022%202%20UPMIR.pdf 
Tags: 
Add Tag
No Tags, Be the first to tag this record!

Summary:  Recent breakthroughs with numerous visual experiences using mobile devices
encourage the research of humancomputer interaction (HCI) involving hand
gesture recognition for Holograms, Virtual Reality, and Augmented Reality.
The rise of these technologies allows educators in medical segments to apply
new pedagogy by interacting with virtual content in a coherent learning
environment. In this thesis, the Central Nervous System (CNS) interaction
is implemented using the Skeleton Joints Moment (SJM) approach for data
reduction and convex hull k Nearest Neighbour (Convex Hull kNN) for hand
gesture classification. Principal Component Analysis (PCA) is commonly
used for dimensional reduction as a data preprocessing for machine learning
like kNN, Support Vector Machine (SVM), and Artificial Neural Network.
However, PCA implementation requires recalculation for a new batch of data.
Therefore, this thesis presented the SJM CH kNN with the Density Mapping
to classify hand gestures in CNS application. Evaluation results show
that this method supports incremental learning with optimized classification
complexity than PCA kNN, SVM, and ANN.
This thesis introduces SJM CH kNN with Density Mapping that addresses
three hologram interaction issues using lowend mobile devices. The issues
are data dimensionality, the complexity of hand gesture classification for incremental
learning, and the uncertainty of hand gesture classification within a class intersection. First, this thesis proposed a robust centroid moment technique for hand gesture
data to reduce kdimensional space to achieve significant data reduction
while retaining hand gesture information. SJM reduces k dimensional data
from hand gesture skeleton data to three principal components (x, y, and zaxis).
These components represent hand gesture moments. Researchers have
proposed different methods of data reduction. One of the methods is PCA.
PCA technique has similar accuracy compared to SJM. However, when new
data is inserted, PCA must decompose the large datasets into a matrix of
eigenvector and eigenvalue to describe their magnitude. Evaluation results
using kNN show that SJM has better accuracy than PCA for skeleton data.
PCA has a higher uncertainty of mean error of 0.75 compared to SJM at only
0.01. In terms of accuracy, SJM shows 96% of prediction accuracy, similar to
PCA using hand skeleton joints but with O(n) complexity compared to PCA
with O(min(p3, n3)) where n is the data points in the dataset, and p is the
features.
Secondly, the importance and originality of this study are that it explores
the complexity of hand gesture classification for incremental learning using a
lowend device. Thus, this thesis presented a Convex Hull kNN approach to
optimize hand gesture classification complexity. The advantage of traditional
kNN is that it does not require preprocessing for a new batch of data. Many
researchers have utilized kNN to classify hand gestures in the past decades
before moving to SVM and ANN. However, it is not practical for big data
where the complexity is O(n). The solution is to extend the Convex Hull
method into kNN. The kvalue is the smallest intersected region of hand
gesture classes. The evaluation result of the ttest shows that P < 0.05 where
there is a significant difference between Convex Hull SJM and Convex Hull
PCA. Thus, the SJM is feasible for Convex Hull SJM kNN and has the
complexity of classification of O(c ∗ log(c)) for none intersected regions which
is better than traditional kNN O(n) and ANN O(nt ∗ (ij + jk + kl)) where
i, j, k, and l are nodes, with t training examples and n epochs, and SVM
O(n3). The experiment shows that SJM CH kNN optimized computational
complexity by O(c + i ∗ log(c + i)) for an incremental dataset in a realtime
environment with high accuracy of 98% where c is the convex hull points.
Finally, the third aim of this study is to investigate the uncertainty of hand
gesture classification. The primary concern of Convex Hull SJM kNN is
the intersected region of the convex hull. Therefore in this thesis, density
mapping for the intersected convex hulls is introduced in the CNS system.
Using Convex Hull SJM kNN with density mapping reduces 76% of data.
The ttest result shows that SJM data and SJM Density Mapping data have
a significant difference of P < 0.05. The F1 score results of this experiment
show that Convex Hull SJM kNN with Density Mapping has 94% of accuracy.
The results indicate that Density Mapping reduces the data size into a fixed
data frame for intersected convex hulls. 
