Data filtering framework for preserving meaningful data records from streams of unstructured weather data

The aim of this research is to design and implement a data filtering framework for preserving meaningful data records from streams of unstructured weather data based on data collection, data pre-process, data filtering, classification, and visualization. The data collection involved monitoring data...

Full description

Saved in:

Bibliographic Details
Main Author:	Al-Zyadat, Wa'el Jum'ah
Format:	Thesis
Language:	English
Published:	2014
Subjects:	Data structures (Computer science) Weather - Data processing
Online Access:	http://psasir.upm.edu.my/id/eprint/60463/1/FSKTM%202014%2017IR.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-upm-ir.60463
record_format	uketd_dc
spelling	my-upm-ir.604632018-05-08T07:50:49Z Data filtering framework for preserving meaningful data records from streams of unstructured weather data 2014-07 Al-Zyadat, Wa'el Jum'ah The aim of this research is to design and implement a data filtering framework for preserving meaningful data records from streams of unstructured weather data based on data collection, data pre-process, data filtering, classification, and visualization. The data collection involved monitoring data and data structuring; data pre-process, error checking, error validation, and correction of error; data filtering involved filtering concept, sequential process, and particles filtering while classification data involved the proposed 5M method. The environmental data creates challenges in the context of storage and data processing, which producing large volume of data collection including a massive 30% of unwanted data. It affects the correctness of data in term of usage The rationale and importance of this research comes in forms of creating step by step operations of data filtering method using local weather data that combines data preprocessing and data filtering steps to enhance the accuracy of data classifications. This research successfully reformatted data collected from sensor-boards which are unstructured, with features that structure raw data to a standardized data format. The analysis of the proposed framework employed three measurements approaches for the validation purposes. First, data pre-process component measurement using correctness and precision measures; filtering components are measured by data (indexing and item) using the context of records to discover the duplicates between two different datasets, and finally the 5M classification is measured by the percentage of total and meaningful_ sensitivity to indicated classification of relevant items which explore the meaningful data of measurements based on percentages of classifier data from relevant data. Result shows the pre-processed data collected is reduced by 69.23 % with similar accuracy as compared to the raw data. Classification hit is 84.6 % accuracy to each clustered data. This research has been able to classify streaming weather data. The execution of data filtering framework for preserving meaningful data records from streams of unstructured weather data produced highly accurate classification clusters. Data structures (Computer science) Weather - Data processing 2014-07 Thesis http://psasir.upm.edu.my/id/eprint/60463/ http://psasir.upm.edu.my/id/eprint/60463/1/FSKTM%202014%2017IR.pdf text en public doctoral Universiti Putra Malaysia Data structures (Computer science) Weather - Data processing
institution	Universiti Putra Malaysia
collection	PSAS Institutional Repository
language	English
topic	Data structures (Computer science) Weather - Data processing
spellingShingle	Data structures (Computer science) Weather - Data processing Al-Zyadat, Wa'el Jum'ah Data filtering framework for preserving meaningful data records from streams of unstructured weather data
description	The aim of this research is to design and implement a data filtering framework for preserving meaningful data records from streams of unstructured weather data based on data collection, data pre-process, data filtering, classification, and visualization. The data collection involved monitoring data and data structuring; data pre-process, error checking, error validation, and correction of error; data filtering involved filtering concept, sequential process, and particles filtering while classification data involved the proposed 5M method. The environmental data creates challenges in the context of storage and data processing, which producing large volume of data collection including a massive 30% of unwanted data. It affects the correctness of data in term of usage The rationale and importance of this research comes in forms of creating step by step operations of data filtering method using local weather data that combines data preprocessing and data filtering steps to enhance the accuracy of data classifications. This research successfully reformatted data collected from sensor-boards which are unstructured, with features that structure raw data to a standardized data format. The analysis of the proposed framework employed three measurements approaches for the validation purposes. First, data pre-process component measurement using correctness and precision measures; filtering components are measured by data (indexing and item) using the context of records to discover the duplicates between two different datasets, and finally the 5M classification is measured by the percentage of total and meaningful_ sensitivity to indicated classification of relevant items which explore the meaningful data of measurements based on percentages of classifier data from relevant data. Result shows the pre-processed data collected is reduced by 69.23 % with similar accuracy as compared to the raw data. Classification hit is 84.6 % accuracy to each clustered data. This research has been able to classify streaming weather data. The execution of data filtering framework for preserving meaningful data records from streams of unstructured weather data produced highly accurate classification clusters.
format	Thesis
qualification_level	Doctorate
author	Al-Zyadat, Wa'el Jum'ah
author_facet	Al-Zyadat, Wa'el Jum'ah
author_sort	Al-Zyadat, Wa'el Jum'ah
title	Data filtering framework for preserving meaningful data records from streams of unstructured weather data
title_short	Data filtering framework for preserving meaningful data records from streams of unstructured weather data
title_full	Data filtering framework for preserving meaningful data records from streams of unstructured weather data
title_fullStr	Data filtering framework for preserving meaningful data records from streams of unstructured weather data
title_full_unstemmed	Data filtering framework for preserving meaningful data records from streams of unstructured weather data
title_sort	data filtering framework for preserving meaningful data records from streams of unstructured weather data
granting_institution	Universiti Putra Malaysia
publishDate	2014
url	http://psasir.upm.edu.my/id/eprint/60463/1/FSKTM%202014%2017IR.pdf
_version_	1747812274856263680

Data filtering framework for preserving meaningful data records from streams of unstructured weather data

Similar Items