Residential College | false |
Status | 已發表Published |
Kernel-based Multi-layer Extreme Learning Machines for Representation Learning | |
Wong, C.M.; Vong, C. M.; Wong, P. K.; Cao, J.W. | |
2018-03-01 | |
Source Publication | IEEE Transactions on Neural Networks and Learning Systems (SCI-E) |
ISSN | 2162-237X |
Pages | 757-762 |
Abstract | Recently, multilayer extreme learning machine (MLELM) was applied to stacked autoencoder (SAE) for representation learning. In contrast to traditional SAE, the training time of ML-ELM is significantly reduced from hours to seconds with high accuracy. However, ML-ELM suffers from several drawbacks: i) manual tuning on the number of hidden nodes in every layer is an uncertain factor to training time and generalization; ii) random projection of input weights and bias in every layer of ML-ELM leads to suboptimal model generalization; iii) the pseudo-inverse solution for output weights in every layer incurs relatively large reconstruction error; iv) the storage and execution time for transformation matrices in representation learning is proportional to the number of hidden layers. Inspired by kernel learning, a kernel version of ML-ELM is developed, namely multi-layer kernel ELM (ML-KELM), whose contributions are: i) elimination of manual tuning on the number of hidden nodes in every layer; ii) no random projection mechanism so as to obtain optimal model generalization; iii) exact inverse solution for output weights is guaranteed under invertible kernel matrix, resulting to smaller reconstruction error; iv) all transformation matrices are unified into two matrices only so that storage can be reduced and may shorten model execution time. Benchmark datasets of different sizes have been employed for the evaluation of MLKELM. Experimental results have verified the contributions of the proposed ML-KELM. The improvement in accuracy over benchmark datasets is up to 7%. |
Keyword | Kernel Learning Multilayer Extreme Learning Machine Stacked Auto Encoder Representation Learning |
DOI | 10.1109/TNNLS.2016.2636834 |
Indexed By | SCIE |
Language | 英語English |
The Source to Article | PB_Publication |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | DEPARTMENT OF ELECTROMECHANICAL ENGINEERING |
Corresponding Author | Vong, C. M. |
Recommended Citation GB/T 7714 | Wong, C.M.,Vong, C. M.,Wong, P. K.,et al. Kernel-based Multi-layer Extreme Learning Machines for Representation Learning[J]. IEEE Transactions on Neural Networks and Learning Systems (SCI-E), 2018, 757-762. |
APA | Wong, C.M.., Vong, C. M.., Wong, P. K.., & Cao, J.W. (2018). Kernel-based Multi-layer Extreme Learning Machines for Representation Learning. IEEE Transactions on Neural Networks and Learning Systems (SCI-E), 757-762. |
MLA | Wong, C.M.,et al."Kernel-based Multi-layer Extreme Learning Machines for Representation Learning".IEEE Transactions on Neural Networks and Learning Systems (SCI-E) (2018):757-762. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment