An improvement of back propagation algorithm using halley third order optimisation method for classification problems
Back Propagation (BP) has proven to be a robust algorithm for different connectionist learning problems which commonly available for any functional induction that provides a computationally efficient method. This algorithm utilises first order optimisation method namely Gradient Descent (GD) m...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English English English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | http://eprints.uthm.edu.my/923/1/24p%20NORHAMREEZA%20%20ABDUL%20HAMID.pdf http://eprints.uthm.edu.my/923/2/NORHAMREEZA%20%20ABDUL%20HAMID%20COPYRIGHT%20DECLARATION.pdf http://eprints.uthm.edu.my/923/3/NORHAMREEZA%20%20ABDUL%20HAMID%20WATERMARK.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Back Propagation (BP) has proven to be a robust algorithm for different
connectionist learning problems which commonly available for any functional
induction that provides a computationally efficient method. This algorithm utilises
first order optimisation method namely Gradient Descent (GD) method which
attempts to minimise the error of network. Nevertheless, some major issues need to
be considered. The GD method not performed well in large scale applications and
when higher learning performances are required. Moreover, it has uncertainty in
finding the global minimum of the error function. Besides, they generally depend on
the parameters‟ selections. Thus, improving the BP learning efficiency has become
an important area of research and consideration specifically in optimisation point of
view. The variations of second order optimisation methods have been proposed
which provide less iteration of convergence. Yet, an issue with these methods is
occasionally converging to the undesired local minima. Inspired by the third order
optimisation method which capable to solve unconstrained optimisation problems
efficiently in the mathematical research area, this research endeavours to propose a
new computational Halley method which is third order optimisation in improving the
learning efficiency of BP algorithm namely Halley with Broyden-Fletcher-Goldfarb�Shanno (H-BFGS) and Halley with Davidon-Fletcher-Powell (H-DFP). The
efficiency of the proposed methods is compared with the first and second order
optimisation method by means of simulation on UCI Machine Learning Repository,
Knowledge Extraction Evolutionary Learning and Kaggle dataset. The simulation
results show that the highest improvement of H-BFGS in terms of generalisation
accuracy is on the Voice Gender classification with 43.33% improvement for 60:40
data division. While H-DFP, the highest improvement achieved in generalisation
accuracy is on the Seeds classification with 41.73% improvement for 70:30 data
division. Thus, the proposed methods provide significant improvement and
promising result in learning Artificial Neural Networks. |
|---|