UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System
Jie Du1; Peng Liu2; Chi Man Vong2; Chuangquan Chen3; Tianfu Wang1; C. L.P. Chen4
2024-08
Source PublicationIEEE Transactions on Neural Networks and Learning Systems
ISSN2162-237X
Volume35Issue:8Pages:11332-11345
Abstract

Machine learning aims to generate a predictive model from a training dataset of a fixed number of known classes. However, many real-world applications (such as health monitoring and elderly care) are data streams in which new data arrive continually in a short time. Such new data may even belong to previously unknown classes. Hence, class-incremental learning (CIL) is necessary, which incrementally and rapidly updates an existing model with the data of new classes while retaining the existing knowledge of old classes. However, most current CIL methods are designed based on deep models that require a computationally expensive training and update process. In addition, deep learning based CIL (DCIL) methods typically employ stochastic gradient descent (SGD) as an optimizer that forgets the old knowledge to a certain extent. In this article, a broad learning system-based CIL (BLS-CIL) method with fast update and high retainability of old class knowledge is proposed. Traditional BLS is a fast and effective shallow neural network, but it does not work well on CIL tasks. However, our proposed BLS-CIL can overcome these issues and provide the following: 1) high accuracy due to our novel class-correlation loss function that considers the correlations between old and new classes; 2) significantly short training/update time due to the newly derived closed-form solution for our class-correlation loss without iterative optimization; and 3) high retainability of old class knowledge due to our newly derived recursive update rule for CIL (RULL) that does not replay the exemplars of all old classes, as contrasted to the exemplars-replaying methods with the SGD optimizer. The proposed BLS-CIL has been evaluated over 12 real-world datasets, including seven tabular/numerical datasets and six image datasets, and the compared methods include one shallow network and seven classical or state-of-the-art DCIL methods. Experimental results show that our BIL-CIL can significantly improve the classification performance over a shallow network by a large margin (8.80%–48.42%). It also achieves comparable or even higher accuracy than DCIL methods, but greatly reduces the training time from hours to minutes and the update time from minutes to seconds.

KeywordBroad Learning System (Bls) Catastrophic Forgetting Class Correlations Class-incremental Learning (Cil) Recursive Update Rule
DOI10.1109/TNNLS.2023.3259016
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS IDWOS:000965742800001
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141
Scopus ID2-s2.0-85151551894
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorPeng Liu; Chi Man Vong
Affiliation1.Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Shenzhen University, Shenzhen, China
2.Department of Computer and Information Science, University of Macau, SAR, China
3.Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, China
4.School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Jie Du,Peng Liu,Chi Man Vong,et al. Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(8), 11332-11345.
APA Jie Du., Peng Liu., Chi Man Vong., Chuangquan Chen., Tianfu Wang., & C. L.P. Chen (2024). Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System. IEEE Transactions on Neural Networks and Learning Systems, 35(8), 11332-11345.
MLA Jie Du,et al."Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System".IEEE Transactions on Neural Networks and Learning Systems 35.8(2024):11332-11345.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Jie Du]'s Articles
[Peng Liu]'s Articles
[Chi Man Vong]'s Articles
Baidu academic
Similar articles in Baidu academic
[Jie Du]'s Articles
[Peng Liu]'s Articles
[Chi Man Vong]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Jie Du]'s Articles
[Peng Liu]'s Articles
[Chi Man Vong]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.