Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition

A traditional online handwritten signature recognition system requires direct contact with the acquisition device that may leave a trace on the device's surface. This results in a signature being easily tracked and imitated. Such an acquisition device is not commonly available, while the public...

Full description

Saved in:
Bibliographic Details
Main Author: Khoh, Wee How
Format: Thesis
Published: 2021
Subjects:
Tags: Add Tag
No Tags, Be the first to tag this record!
id my-mmu-ep.11417
record_format uketd_dc
spelling my-mmu-ep.114172023-05-22T04:56:04Z Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition 2021-05 Khoh, Wee How QA75.5-76.95 Electronic computers. Computer science A traditional online handwritten signature recognition system requires direct contact with the acquisition device that may leave a trace on the device's surface. This results in a signature being easily tracked and imitated. Such an acquisition device is not commonly available, while the public usually shares this device. Germs could be accumulated on the devices, and thus, a hygiene concern has risen. A novel approach in recognising a signature based on hand motion is proposed to address these issues, namely in-air hand gesture signature (iHGS). A low-cost acquisition device – the Microsoft Kinect sensor, is used to capture hand gesture-based signatures. Unlike the conventional dynamic signature, the captured hand gesture-based signature is a sequence of images containing the signing action's spatial and temporal information. To detect and extract the region of interest, a hand region is first located and segmented from a depth image by a predictive hand segmentation algorithm. The resultant volume data is then condensed and transformed into three directional plane projections. Specifically, XY plane projection employs Motion History Image (MHI) to obtain a compact motion representation image, whist XT plane projection (X-profile) and YT plane projection (Y-profile) project the volume data along the x-axis and y-axis, respectively. Vector-based and image-based features are extracted from the transformed image templates. A vector-based feature is a one-dimensional vector produced through the image templates into a vector space that numerically quantifies the local information of an image. An image-based feature is a visual representation feature that combines one or more image templates into a static image that better visualizes a hand gesture signature's spatial and temporal information. In the experimental analysis, classification performance and system robustness are systematically assessed using a self-collected dataset, the iHGS dataset. For the classification analysis, the k-NN and SVM classifiers are employed to classify the vector-based features. A pre-trained deep learning model is used to classify imagebased features. On the other hand, system robustness is also investigated against two common forgery attacks, (1) random forgeries and (2) skilled forgeries. Additionally, performance comparisons are conducted between the proposed methods with several state-of-art approaches. The overall performance analysis has demonstrated the potential and efficiency of the proposed methods in recognising and verifying in-air hand gesture signature recognition. 2021-05 Thesis http://shdl.mmu.edu.my/11417/ http://erep.mmu.edu.my/ phd doctoral Multimedia University Faculty of Information Science and Technology (FIST) EREP ID: 9895
institution Multimedia University
collection MMU Institutional Repository
topic QA75.5-76.95 Electronic computers
Computer science
spellingShingle QA75.5-76.95 Electronic computers
Computer science
Khoh, Wee How
Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
description A traditional online handwritten signature recognition system requires direct contact with the acquisition device that may leave a trace on the device's surface. This results in a signature being easily tracked and imitated. Such an acquisition device is not commonly available, while the public usually shares this device. Germs could be accumulated on the devices, and thus, a hygiene concern has risen. A novel approach in recognising a signature based on hand motion is proposed to address these issues, namely in-air hand gesture signature (iHGS). A low-cost acquisition device – the Microsoft Kinect sensor, is used to capture hand gesture-based signatures. Unlike the conventional dynamic signature, the captured hand gesture-based signature is a sequence of images containing the signing action's spatial and temporal information. To detect and extract the region of interest, a hand region is first located and segmented from a depth image by a predictive hand segmentation algorithm. The resultant volume data is then condensed and transformed into three directional plane projections. Specifically, XY plane projection employs Motion History Image (MHI) to obtain a compact motion representation image, whist XT plane projection (X-profile) and YT plane projection (Y-profile) project the volume data along the x-axis and y-axis, respectively. Vector-based and image-based features are extracted from the transformed image templates. A vector-based feature is a one-dimensional vector produced through the image templates into a vector space that numerically quantifies the local information of an image. An image-based feature is a visual representation feature that combines one or more image templates into a static image that better visualizes a hand gesture signature's spatial and temporal information. In the experimental analysis, classification performance and system robustness are systematically assessed using a self-collected dataset, the iHGS dataset. For the classification analysis, the k-NN and SVM classifiers are employed to classify the vector-based features. A pre-trained deep learning model is used to classify imagebased features. On the other hand, system robustness is also investigated against two common forgery attacks, (1) random forgeries and (2) skilled forgeries. Additionally, performance comparisons are conducted between the proposed methods with several state-of-art approaches. The overall performance analysis has demonstrated the potential and efficiency of the proposed methods in recognising and verifying in-air hand gesture signature recognition.
format Thesis
qualification_name Doctor of Philosophy (PhD.)
qualification_level Doctorate
author Khoh, Wee How
author_facet Khoh, Wee How
author_sort Khoh, Wee How
title Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
title_short Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
title_full Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
title_fullStr Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
title_full_unstemmed Spatial-Temporal Analysis of In-Air Hand Gesture Signature Recognition
title_sort spatial-temporal analysis of in-air hand gesture signature recognition
granting_institution Multimedia University
granting_department Faculty of Information Science and Technology (FIST)
publishDate 2021
_version_ 1776101403618443264