Enhanced extreme learning machine for general regression and classification tasks
Extreme Learning Machine (ELM) is a single hidden layer feedforward neural network which randomly chooses hidden nodes and analytically determines the output weights using least square method. Despite its popularity, ELM has a number of challenges worth to investigating for improving the usabilit...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/92877/1/FK%202020%2097%20-%20IR.1.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Extreme Learning Machine (ELM) is a single hidden layer feedforward neural
network which randomly chooses hidden nodes and analytically determines
the output weights using least square method. Despite its popularity, ELM has
a number of challenges worth to investigating for improving the usability of
ELM in a more advanced application. This thesis focusses on challenges
namely design architecture and learning technique. The first challenge is to
select the optimal number of hidden nodes for ELM in different application. To
address this problem, a new approach referred to SVM-ELM is proposed,
which utilizes 1-norm support vector machine (SVM) to the hidden layer matrix
of ELM in order to automatically discover the optimal number of hidden nodes.
The method is developed for regression task by using mean/ median of ELM
training errors which is then used as threshold for separating the training data
and converting the continuous targets to binary. This will allow projection to 1-
norm SVM dimension in order to find the best number of nodes as support
vectors. Second problem in ELM, is the restriction in performance of ELM in
terms of training time and model generalization, due to the complexity of
singular value decomposition (SVD) for computing the Moore-Penrose
generalized inverse of the hidden layer matrix, especially on a large matrix. To
address this issue, a fast adaptive shrinkage/thresholding algorithm ELM
(FASTA-ELM) which uses an extension of forward-backward splitting (FBS) to
compute the smallest norm of the output weights in ELM is presented. The
proposed FASTA-ELM replaces the analytical step usually solved by SVD with
an approximate solution through proximal gradient method, which dramatically
speeds up the training time and improves the generalization ability in
classification task. The performance of FASTA-ELM is evaluated on face
gender recognition problem and the result is comparable to other state-of-theart
methods, with significantly reduced training time. For instance, the training time of 1000 nodes ELM is 18.11 s, while FASTA-ELM completed in 1.671 s.
The proposed modification to the ELM shows significant improvement to the
conventional ELM in terms of training time and accuracy, and provide good
generalization performance in regression and classification tasks. |
|---|