Enhanced extreme learning machine for general regression and classification tasks

Extreme Learning Machine (ELM) is a single hidden layer feedforward neural network which randomly chooses hidden nodes and analytically determines the output weights using least square method. Despite its popularity, ELM has a number of challenges worth to investigating for improving the usabilit...

Full description

Saved in:

Bibliographic Details
Main Author:	Mahmood, Saif F
Format:	Thesis
Language:	English
Published:	2020
Subjects:	Artificial intelligence - Engineering applications Machine learning Regression analysis
Online Access:	http://psasir.upm.edu.my/id/eprint/92877/1/FK%202020%2097%20-%20IR.1.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-upm-ir.92877
record_format	uketd_dc
spelling	my-upm-ir.928772022-10-17T07:22:09Z Enhanced extreme learning machine for general regression and classification tasks 2020-09 Mahmood, Saif F Extreme Learning Machine (ELM) is a single hidden layer feedforward neural network which randomly chooses hidden nodes and analytically determines the output weights using least square method. Despite its popularity, ELM has a number of challenges worth to investigating for improving the usability of ELM in a more advanced application. This thesis focusses on challenges namely design architecture and learning technique. The first challenge is to select the optimal number of hidden nodes for ELM in different application. To address this problem, a new approach referred to SVM-ELM is proposed, which utilizes 1-norm support vector machine (SVM) to the hidden layer matrix of ELM in order to automatically discover the optimal number of hidden nodes. The method is developed for regression task by using mean/ median of ELM training errors which is then used as threshold for separating the training data and converting the continuous targets to binary. This will allow projection to 1- norm SVM dimension in order to find the best number of nodes as support vectors. Second problem in ELM, is the restriction in performance of ELM in terms of training time and model generalization, due to the complexity of singular value decomposition (SVD) for computing the Moore-Penrose generalized inverse of the hidden layer matrix, especially on a large matrix. To address this issue, a fast adaptive shrinkage/thresholding algorithm ELM (FASTA-ELM) which uses an extension of forward-backward splitting (FBS) to compute the smallest norm of the output weights in ELM is presented. The proposed FASTA-ELM replaces the analytical step usually solved by SVD with an approximate solution through proximal gradient method, which dramatically speeds up the training time and improves the generalization ability in classification task. The performance of FASTA-ELM is evaluated on face gender recognition problem and the result is comparable to other state-of-theart methods, with significantly reduced training time. For instance, the training time of 1000 nodes ELM is 18.11 s, while FASTA-ELM completed in 1.671 s. The proposed modification to the ELM shows significant improvement to the conventional ELM in terms of training time and accuracy, and provide good generalization performance in regression and classification tasks. Artificial intelligence - Engineering applications Machine learning Regression analysis 2020-09 Thesis http://psasir.upm.edu.my/id/eprint/92877/ http://psasir.upm.edu.my/id/eprint/92877/1/FK%202020%2097%20-%20IR.1.pdf text en public doctoral Universiti Putra Malaysia Artificial intelligence - Engineering applications Machine learning Regression analysis Marhaban, Mohammad Hamiruce
institution	Universiti Putra Malaysia
collection	PSAS Institutional Repository
language	English
advisor	Marhaban, Mohammad Hamiruce
topic	Artificial intelligence - Engineering applications Machine learning Regression analysis
spellingShingle	Artificial intelligence - Engineering applications Machine learning Regression analysis Mahmood, Saif F Enhanced extreme learning machine for general regression and classification tasks
description	Extreme Learning Machine (ELM) is a single hidden layer feedforward neural network which randomly chooses hidden nodes and analytically determines the output weights using least square method. Despite its popularity, ELM has a number of challenges worth to investigating for improving the usability of ELM in a more advanced application. This thesis focusses on challenges namely design architecture and learning technique. The first challenge is to select the optimal number of hidden nodes for ELM in different application. To address this problem, a new approach referred to SVM-ELM is proposed, which utilizes 1-norm support vector machine (SVM) to the hidden layer matrix of ELM in order to automatically discover the optimal number of hidden nodes. The method is developed for regression task by using mean/ median of ELM training errors which is then used as threshold for separating the training data and converting the continuous targets to binary. This will allow projection to 1- norm SVM dimension in order to find the best number of nodes as support vectors. Second problem in ELM, is the restriction in performance of ELM in terms of training time and model generalization, due to the complexity of singular value decomposition (SVD) for computing the Moore-Penrose generalized inverse of the hidden layer matrix, especially on a large matrix. To address this issue, a fast adaptive shrinkage/thresholding algorithm ELM (FASTA-ELM) which uses an extension of forward-backward splitting (FBS) to compute the smallest norm of the output weights in ELM is presented. The proposed FASTA-ELM replaces the analytical step usually solved by SVD with an approximate solution through proximal gradient method, which dramatically speeds up the training time and improves the generalization ability in classification task. The performance of FASTA-ELM is evaluated on face gender recognition problem and the result is comparable to other state-of-theart methods, with significantly reduced training time. For instance, the training time of 1000 nodes ELM is 18.11 s, while FASTA-ELM completed in 1.671 s. The proposed modification to the ELM shows significant improvement to the conventional ELM in terms of training time and accuracy, and provide good generalization performance in regression and classification tasks.
format	Thesis
qualification_level	Doctorate
author	Mahmood, Saif F
author_facet	Mahmood, Saif F
author_sort	Mahmood, Saif F
title	Enhanced extreme learning machine for general regression and classification tasks
title_short	Enhanced extreme learning machine for general regression and classification tasks
title_full	Enhanced extreme learning machine for general regression and classification tasks
title_fullStr	Enhanced extreme learning machine for general regression and classification tasks
title_full_unstemmed	Enhanced extreme learning machine for general regression and classification tasks
title_sort	enhanced extreme learning machine for general regression and classification tasks
granting_institution	Universiti Putra Malaysia
publishDate	2020
url	http://psasir.upm.edu.my/id/eprint/92877/1/FK%202020%2097%20-%20IR.1.pdf
_version_	1747813775904342016

Enhanced extreme learning machine for general regression and classification tasks

Similar Items