Residential College | false |
Status | 已發表Published |
Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System | |
Jie Du1; Peng Liu2; Chi Man Vong2; Chuangquan Chen3; Tianfu Wang1; C. L.P. Chen4 | |
2024-08 | |
Source Publication | IEEE Transactions on Neural Networks and Learning Systems |
ISSN | 2162-237X |
Volume | 35Issue:8Pages:11332-11345 |
Abstract | Machine learning aims to generate a predictive model from a training dataset of a fixed number of known classes. However, many real-world applications (such as health monitoring and elderly care) are data streams in which new data arrive continually in a short time. Such new data may even belong to previously unknown classes. Hence, class-incremental learning (CIL) is necessary, which incrementally and rapidly updates an existing model with the data of new classes while retaining the existing knowledge of old classes. However, most current CIL methods are designed based on deep models that require a computationally expensive training and update process. In addition, deep learning based CIL (DCIL) methods typically employ stochastic gradient descent (SGD) as an optimizer that forgets the old knowledge to a certain extent. In this article, a broad learning system-based CIL (BLS-CIL) method with fast update and high retainability of old class knowledge is proposed. Traditional BLS is a fast and effective shallow neural network, but it does not work well on CIL tasks. However, our proposed BLS-CIL can overcome these issues and provide the following: 1) high accuracy due to our novel class-correlation loss function that considers the correlations between old and new classes; 2) significantly short training/update time due to the newly derived closed-form solution for our class-correlation loss without iterative optimization; and 3) high retainability of old class knowledge due to our newly derived recursive update rule for CIL (RULL) that does not replay the exemplars of all old classes, as contrasted to the exemplars-replaying methods with the SGD optimizer. The proposed BLS-CIL has been evaluated over 12 real-world datasets, including seven tabular/numerical datasets and six image datasets, and the compared methods include one shallow network and seven classical or state-of-the-art DCIL methods. Experimental results show that our BIL-CIL can significantly improve the classification performance over a shallow network by a large margin (8.80%–48.42%). It also achieves comparable or even higher accuracy than DCIL methods, but greatly reduces the training time from hours to minutes and the update time from minutes to seconds. |
Keyword | Broad Learning System (Bls) Catastrophic Forgetting Class Correlations Class-incremental Learning (Cil) Recursive Update Rule |
DOI | 10.1109/TNNLS.2023.3259016 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Computer Science ; Engineering |
WOS Subject | Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
WOS ID | WOS:000965742800001 |
Publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 445 HOES LANE, PISCATAWAY, NJ 08855-4141 |
Scopus ID | 2-s2.0-85151551894 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Peng Liu; Chi Man Vong |
Affiliation | 1.Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Shenzhen University, Shenzhen, China 2.Department of Computer and Information Science, University of Macau, SAR, China 3.Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, China 4.School of Computer Science and Engineering, South China University of Technology, Guangzhou, China |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Jie Du,Peng Liu,Chi Man Vong,et al. Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(8), 11332-11345. |
APA | Jie Du., Peng Liu., Chi Man Vong., Chuangquan Chen., Tianfu Wang., & C. L.P. Chen (2024). Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System. IEEE Transactions on Neural Networks and Learning Systems, 35(8), 11332-11345. |
MLA | Jie Du,et al."Class-Incremental Learning Method With Fast Update and High Retainability Based on Broad Learning System".IEEE Transactions on Neural Networks and Learning Systems 35.8(2024):11332-11345. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment