Hand gesture classification using convex hull skeleton joints moment k-nn technique for dimensionality reduction and incremental learning in the central nervous system’s hologram
Recent breakthroughs with numerous visual experiences using mobile devices encourage the research of human-computer interaction (HCI) involving hand gesture recognition for Holograms, Virtual Reality, and Augmented Reality. The rise of these technologies allows educators in medical segments to ap...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/104004/1/FSKTM%202022%202%20UPMIR.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Recent breakthroughs with numerous visual experiences using mobile devices
encourage the research of human-computer interaction (HCI) involving hand
gesture recognition for Holograms, Virtual Reality, and Augmented Reality.
The rise of these technologies allows educators in medical segments to apply
new pedagogy by interacting with virtual content in a coherent learning
environment. In this thesis, the Central Nervous System (CNS) interaction
is implemented using the Skeleton Joints Moment (SJM) approach for data
reduction and convex hull k Nearest Neighbour (Convex Hull k-NN) for hand
gesture classification. Principal Component Analysis (PCA) is commonly
used for dimensional reduction as a data preprocessing for machine learning
like k-NN, Support Vector Machine (SVM), and Artificial Neural Network.
However, PCA implementation requires recalculation for a new batch of data.
Therefore, this thesis presented the SJM CH k-NN with the Density Mapping
to classify hand gestures in CNS application. Evaluation results show
that this method supports incremental learning with optimized classification
complexity than PCA k-NN, SVM, and ANN.
This thesis introduces SJM CH k-NN with Density Mapping that addresses
three hologram interaction issues using low-end mobile devices. The issues
are data dimensionality, the complexity of hand gesture classification for incremental
learning, and the uncertainty of hand gesture classification within a class intersection. First, this thesis proposed a robust centroid moment technique for hand gesture
data to reduce k-dimensional space to achieve significant data reduction
while retaining hand gesture information. SJM reduces k dimensional data
from hand gesture skeleton data to three principal components (x, y, and zaxis).
These components represent hand gesture moments. Researchers have
proposed different methods of data reduction. One of the methods is PCA.
PCA technique has similar accuracy compared to SJM. However, when new
data is inserted, PCA must decompose the large datasets into a matrix of
eigenvector and eigenvalue to describe their magnitude. Evaluation results
using k-NN show that SJM has better accuracy than PCA for skeleton data.
PCA has a higher uncertainty of mean error of 0.75 compared to SJM at only
0.01. In terms of accuracy, SJM shows 96% of prediction accuracy, similar to
PCA using hand skeleton joints but with O(n) complexity compared to PCA
with O(min(p3, n3)) where n is the data points in the dataset, and p is the
features.
Secondly, the importance and originality of this study are that it explores
the complexity of hand gesture classification for incremental learning using a
low-end device. Thus, this thesis presented a Convex Hull k-NN approach to
optimize hand gesture classification complexity. The advantage of traditional
k-NN is that it does not require preprocessing for a new batch of data. Many
researchers have utilized k-NN to classify hand gestures in the past decades
before moving to SVM and ANN. However, it is not practical for big data
where the complexity is O(n). The solution is to extend the Convex Hull
method into k-NN. The k-value is the smallest intersected region of hand
gesture classes. The evaluation result of the t-test shows that P < 0.05 where
there is a significant difference between Convex Hull SJM and Convex Hull
PCA. Thus, the SJM is feasible for Convex Hull SJM k-NN and has the
complexity of classification of O(c ∗ log(c)) for none intersected regions which
is better than traditional k-NN O(n) and ANN O(nt ∗ (ij + jk + kl)) where
i, j, k, and l are nodes, with t training examples and n epochs, and SVM
O(n3). The experiment shows that SJM CH k-NN optimized computational
complexity by O(c + i ∗ log(c + i)) for an incremental dataset in a real-time
environment with high accuracy of 98% where c is the convex hull points.
Finally, the third aim of this study is to investigate the uncertainty of hand
gesture classification. The primary concern of Convex Hull SJM k-NN is
the intersected region of the convex hull. Therefore in this thesis, density
mapping for the intersected convex hulls is introduced in the CNS system.
Using Convex Hull SJM k-NN with density mapping reduces 76% of data.
The t-test result shows that SJM data and SJM Density Mapping data have
a significant difference of P < 0.05. The F1 score results of this experiment
show that Convex Hull SJM k-NN with Density Mapping has 94% of accuracy.
The results indicate that Density Mapping reduces the data size into a fixed
data frame for intersected convex hulls. |
|---|