Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer

The rise of Industrial Revolution 4.0 (IR4.0) technology, such as the Internet of Things (IoT), leads to the existence of massive volumes of data. The phenomenon produces a vast volume and variety of data and increases production speed. Consequently, to handle these data, computer algorithms must ad...

Full description

Saved in:

Bibliographic Details
Main Author:	Iqbal Basheer, Muhammmad Yunus
Format:	Thesis
Language:	English
Published:	2023
Subjects:	Streaming technology (Telecommunications)
Online Access:	https://ir.uitm.edu.my/id/eprint/91110/1/91110.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-uitm-ir.91110
record_format	uketd_dc
spelling	my-uitm-ir.911102024-07-22T04:22:34Z Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer 2023 Iqbal Basheer, Muhammmad Yunus Streaming technology (Telecommunications) The rise of Industrial Revolution 4.0 (IR4.0) technology, such as the Internet of Things (IoT), leads to the existence of massive volumes of data. The phenomenon produces a vast volume and variety of data and increases production speed. Consequently, to handle these data, computer algorithms must adapt to their characteristics. Due to its massive volume, variety, and velocity, it contains a lot of insightful patterns. These patterns may include both normal and anomalies data. Anomalies are important to be detected as its existence may require immediate attention and actions. The anomaly data deviate far from normal and may feed wrong information that might lead to wrong decisions and predictions. Hence, it is critical for an anomaly detection algorithm to detect data anomalies patterns. Nonetheless, the process of detecting anomalies in streaming data is laborious. The available algorithms will face difficulties due to the abundance of data produced over time. Furthermore, it needs to operate fast. This research focuses on anomaly detection in streaming data. We built three algorithms to detect anomalies in the streaming data autonomously. These algorithms are data-driven and do not require thresholds or predefined assumptions. They are nonparametric and have no assumptions on the distribution of data. Autonomous anomaly detection (AAD) is enhanced to receive streaming data. It is called multithreaded autonomous anomaly detection for streaming data (MAAD) which works asynchronously while using recursive updates to calculate required mechanisms such as mean and average scalar products. After that, autonomous anomaly detection for streaming data (AADS) is proposed to detect anomalies in any amount of data. The AADS algorithm uses evolving methods which are evolving autonomous data partitioning (eADP) and non-weighted frequency equations. Finally, the AADS is enhanced to operate parallelly, called parallel autonomous anomaly detection for streaming data (PAADS). It is because the parallel mechanism is able to handle high-speed streaming data. The proposed algorithms were evaluated to test their speed in handling streaming data. The performance tests are also conducted to assess whether each algorithm can detect most of the true anomalies. The data is supplied using IoT devices, and benchmark datasets are also presented to test the algorithm's performance. 2023 Thesis https://ir.uitm.edu.my/id/eprint/91110/ https://ir.uitm.edu.my/id/eprint/91110/1/91110.pdf text en public masters Universiti Teknologi MARA (UiTM) College of Computing, Informatics and Mathematics Mohd Ali, Azliza
institution	Universiti Teknologi MARA
collection	UiTM Institutional Repository
language	English
advisor	Mohd Ali, Azliza
topic	Streaming technology (Telecommunications)
spellingShingle	Streaming technology (Telecommunications) Iqbal Basheer, Muhammmad Yunus Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
description	The rise of Industrial Revolution 4.0 (IR4.0) technology, such as the Internet of Things (IoT), leads to the existence of massive volumes of data. The phenomenon produces a vast volume and variety of data and increases production speed. Consequently, to handle these data, computer algorithms must adapt to their characteristics. Due to its massive volume, variety, and velocity, it contains a lot of insightful patterns. These patterns may include both normal and anomalies data. Anomalies are important to be detected as its existence may require immediate attention and actions. The anomaly data deviate far from normal and may feed wrong information that might lead to wrong decisions and predictions. Hence, it is critical for an anomaly detection algorithm to detect data anomalies patterns. Nonetheless, the process of detecting anomalies in streaming data is laborious. The available algorithms will face difficulties due to the abundance of data produced over time. Furthermore, it needs to operate fast. This research focuses on anomaly detection in streaming data. We built three algorithms to detect anomalies in the streaming data autonomously. These algorithms are data-driven and do not require thresholds or predefined assumptions. They are nonparametric and have no assumptions on the distribution of data. Autonomous anomaly detection (AAD) is enhanced to receive streaming data. It is called multithreaded autonomous anomaly detection for streaming data (MAAD) which works asynchronously while using recursive updates to calculate required mechanisms such as mean and average scalar products. After that, autonomous anomaly detection for streaming data (AADS) is proposed to detect anomalies in any amount of data. The AADS algorithm uses evolving methods which are evolving autonomous data partitioning (eADP) and non-weighted frequency equations. Finally, the AADS is enhanced to operate parallelly, called parallel autonomous anomaly detection for streaming data (PAADS). It is because the parallel mechanism is able to handle high-speed streaming data. The proposed algorithms were evaluated to test their speed in handling streaming data. The performance tests are also conducted to assess whether each algorithm can detect most of the true anomalies. The data is supplied using IoT devices, and benchmark datasets are also presented to test the algorithm's performance.
format	Thesis
qualification_level	Master's degree
author	Iqbal Basheer, Muhammmad Yunus
author_facet	Iqbal Basheer, Muhammmad Yunus
author_sort	Iqbal Basheer, Muhammmad Yunus
title	Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
title_short	Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
title_full	Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
title_fullStr	Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
title_full_unstemmed	Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer
title_sort	autonomous anomaly detection using density-based features in streaming data / muhammmad yunus iqbal basheer
granting_institution	Universiti Teknologi MARA (UiTM)
granting_department	College of Computing, Informatics and Mathematics
publishDate	2023
url	https://ir.uitm.edu.my/id/eprint/91110/1/91110.pdf
_version_	1811768758026371072

Autonomous anomaly detection using density-based features in streaming data / Muhammmad Yunus Iqbal Basheer

Similar Items