An improvement of back propagation algorithm using halley third order optimisation method for classification problems

Back Propagation (BP) has proven to be a robust algorithm for different connectionist learning problems which commonly available for any functional induction that provides a computationally efficient method. This algorithm utilises first order optimisation method namely Gradient Descent (GD) m...

Full description

Saved in:

Bibliographic Details
Main Author:	Abdul Hamid, Norhamreeza
Format:	Thesis
Language:	English English English
Published:	2020
Subjects:	QA75-76.95 Calculating machines
Online Access:	http://eprints.uthm.edu.my/923/1/24p%20NORHAMREEZA%20%20ABDUL%20HAMID.pdf http://eprints.uthm.edu.my/923/2/NORHAMREEZA%20%20ABDUL%20HAMID%20COPYRIGHT%20DECLARATION.pdf http://eprints.uthm.edu.my/923/3/NORHAMREEZA%20%20ABDUL%20HAMID%20WATERMARK.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-uthm-ep.923
record_format	uketd_dc
spelling	my-uthm-ep.9232021-09-09T04:04:35Z An improvement of back propagation algorithm using halley third order optimisation method for classification problems 2020 Abdul Hamid, Norhamreeza QA75-76.95 Calculating machines Back Propagation (BP) has proven to be a robust algorithm for different connectionist learning problems which commonly available for any functional induction that provides a computationally efficient method. This algorithm utilises first order optimisation method namely Gradient Descent (GD) method which attempts to minimise the error of network. Nevertheless, some major issues need to be considered. The GD method not performed well in large scale applications and when higher learning performances are required. Moreover, it has uncertainty in finding the global minimum of the error function. Besides, they generally depend on the parameters‟ selections. Thus, improving the BP learning efficiency has become an important area of research and consideration specifically in optimisation point of view. The variations of second order optimisation methods have been proposed which provide less iteration of convergence. Yet, an issue with these methods is occasionally converging to the undesired local minima. Inspired by the third order optimisation method which capable to solve unconstrained optimisation problems efficiently in the mathematical research area, this research endeavours to propose a new computational Halley method which is third order optimisation in improving the learning efficiency of BP algorithm namely Halley with Broyden-Fletcher-Goldfarb�Shanno (H-BFGS) and Halley with Davidon-Fletcher-Powell (H-DFP). The efficiency of the proposed methods is compared with the first and second order optimisation method by means of simulation on UCI Machine Learning Repository, Knowledge Extraction Evolutionary Learning and Kaggle dataset. The simulation results show that the highest improvement of H-BFGS in terms of generalisation accuracy is on the Voice Gender classification with 43.33% improvement for 60:40 data division. While H-DFP, the highest improvement achieved in generalisation accuracy is on the Seeds classification with 41.73% improvement for 70:30 data division. Thus, the proposed methods provide significant improvement and promising result in learning Artificial Neural Networks. 2020 Thesis http://eprints.uthm.edu.my/923/ http://eprints.uthm.edu.my/923/1/24p%20NORHAMREEZA%20%20ABDUL%20HAMID.pdf text en public http://eprints.uthm.edu.my/923/2/NORHAMREEZA%20%20ABDUL%20HAMID%20COPYRIGHT%20DECLARATION.pdf text en staffonly http://eprints.uthm.edu.my/923/3/NORHAMREEZA%20%20ABDUL%20HAMID%20WATERMARK.pdf text en validuser phd doctoral Universiti Tun Hussein Onn Malaysia Fakulti Sains Komputer dan Teknologi Maklumat
institution	Universiti Tun Hussein Onn Malaysia
collection	UTHM Institutional Repository
language	English English English
topic	QA75-76.95 Calculating machines
spellingShingle	QA75-76.95 Calculating machines Abdul Hamid, Norhamreeza An improvement of back propagation algorithm using halley third order optimisation method for classification problems
description	Back Propagation (BP) has proven to be a robust algorithm for different connectionist learning problems which commonly available for any functional induction that provides a computationally efficient method. This algorithm utilises first order optimisation method namely Gradient Descent (GD) method which attempts to minimise the error of network. Nevertheless, some major issues need to be considered. The GD method not performed well in large scale applications and when higher learning performances are required. Moreover, it has uncertainty in finding the global minimum of the error function. Besides, they generally depend on the parameters‟ selections. Thus, improving the BP learning efficiency has become an important area of research and consideration specifically in optimisation point of view. The variations of second order optimisation methods have been proposed which provide less iteration of convergence. Yet, an issue with these methods is occasionally converging to the undesired local minima. Inspired by the third order optimisation method which capable to solve unconstrained optimisation problems efficiently in the mathematical research area, this research endeavours to propose a new computational Halley method which is third order optimisation in improving the learning efficiency of BP algorithm namely Halley with Broyden-Fletcher-Goldfarb�Shanno (H-BFGS) and Halley with Davidon-Fletcher-Powell (H-DFP). The efficiency of the proposed methods is compared with the first and second order optimisation method by means of simulation on UCI Machine Learning Repository, Knowledge Extraction Evolutionary Learning and Kaggle dataset. The simulation results show that the highest improvement of H-BFGS in terms of generalisation accuracy is on the Voice Gender classification with 43.33% improvement for 60:40 data division. While H-DFP, the highest improvement achieved in generalisation accuracy is on the Seeds classification with 41.73% improvement for 70:30 data division. Thus, the proposed methods provide significant improvement and promising result in learning Artificial Neural Networks.
format	Thesis
qualification_name	Doctor of Philosophy (PhD.)
qualification_level	Doctorate
author	Abdul Hamid, Norhamreeza
author_facet	Abdul Hamid, Norhamreeza
author_sort	Abdul Hamid, Norhamreeza
title	An improvement of back propagation algorithm using halley third order optimisation method for classification problems
title_short	An improvement of back propagation algorithm using halley third order optimisation method for classification problems
title_full	An improvement of back propagation algorithm using halley third order optimisation method for classification problems
title_fullStr	An improvement of back propagation algorithm using halley third order optimisation method for classification problems
title_full_unstemmed	An improvement of back propagation algorithm using halley third order optimisation method for classification problems
title_sort	improvement of back propagation algorithm using halley third order optimisation method for classification problems
granting_institution	Universiti Tun Hussein Onn Malaysia
granting_department	Fakulti Sains Komputer dan Teknologi Maklumat
publishDate	2020
url	http://eprints.uthm.edu.my/923/1/24p%20NORHAMREEZA%20%20ABDUL%20HAMID.pdf http://eprints.uthm.edu.my/923/2/NORHAMREEZA%20%20ABDUL%20HAMID%20COPYRIGHT%20DECLARATION.pdf http://eprints.uthm.edu.my/923/3/NORHAMREEZA%20%20ABDUL%20HAMID%20WATERMARK.pdf
_version_	1747830714818101248

An improvement of back propagation algorithm using halley third order optimisation method for classification problems

Similar Items