Empirical Kernel Map-Based Multilayer Extreme Learning Machines for Representation Learning

UM > Faculty of Science and Technology

Status	已發表Published
	Empirical Kernel Map-Based Multilayer Extreme Learning Machines for Representation Learning
	Vong, C. M.; Chen, C.Q.; Wong, P. K.
	2018-05-01
Source Publication	Neurocomputing (SCI-E)
ISSN	0925-2312
Pages	265-276
Abstract	Recently, multilayer extreme learning machine (ML-ELM) and hierarchical extreme learning machine (H- ELM) were developed for representation learning whose training time can be reduced from hours to seconds compared to traditional stacked autoencoder (SAE). However, there are three practical issues in ML-ELM and H-ELM: (1) the random projection in every layer leads to unstable and suboptimal per- formance; (2) the manual tuning of the number of hidden nodes in every layer is time-consuming; and (3) under large hidden layer, the training time becomes relatively slow and a large storage is necessary. More recently, issues (1) and (2) have been resolved by kernel method, namely, multilayer kernel ELM (ML-KELM), which encodes the hidden layer in form of a kernel matrix (computed by using kernel func- tion on the input data), but the storage and computation issues for kernel matrix pose a big challenge in large-scale application. In this paper, we empirically show that these issues can be alleviated by encod- ing the hidden layer in the form of an approximate empirical kernel map (EKM) computed from low-rank approximation of the kernel matrix. This proposed method is called ML-EKM-ELM, whose contributions are: (1) stable and better performance is achieved under no random projection mechanism; (2) the ex- haustive manual tuning on the number of hidden nodes in every layer is eliminated; (3) EKM is scalable and produces a much smaller hidden layer for fast training and low memory storage, thereby suitable for large-scale problems. Experimental results on benchmark datasets demonstrated the effectiveness of the proposed ML-EKM-ELM. As an illustrative example, on the NORB dataset, ML-EKM-ELM can be re- spectively up to 16 times and 37 times faster than ML-KELM for training and testing with a little loss of accuracy of 0.35%, while the memory storage can be reduced up to 1/9.
Keyword	Kernel learning Multilayer extreme learning machine (ML-ELM) Empirical kernel map (EKM) Representation learning stacked autoencoder (SAE)
Language	英語English
The Source to Article	PB_Publication
PUB ID	36544
Document Type	Journal article
Collection	Faculty of Science and Technology
Corresponding Author	Vong, C. M.
Recommended Citation GB/T 7714	Vong, C. M.,Chen, C.Q.,Wong, P. K.. Empirical Kernel Map-Based Multilayer Extreme Learning Machines for Representation Learning[J]. Neurocomputing (SCI-E), 2018, 265-276.
APA	Vong, C. M.., Chen, C.Q.., & Wong, P. K. (2018). Empirical Kernel Map-Based Multilayer Extreme Learning Machines for Representation Learning. Neurocomputing (SCI-E), 265-276.
MLA	Vong, C. M.,et al."Empirical Kernel Map-Based Multilayer Extreme Learning Machines for Representation Learning".Neurocomputing (SCI-E) (2018):265-276.

Files in This Item:
There are no files associated with this item.

If you have any objections to this item, please fill out the form below and the administrator will contact you as soon as possible.
Content:
Email：	*
Affiliation No.
Verification Code:	Refresh

Any comments and suggestions are welcomed.
Title:	*
Content:
Email：	*
Verification Code:	Refresh