Residential College | false |
Status | 已發表Published |
MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation | |
Pang, Jianhui1; Yang, Baosong2; Wong, Derek Fai1; Liu, Dayiheng2; Wei, Xiangpeng2; Xie, Jun2; Chao, Lidia Sam1 | |
2024-05 | |
Conference Name | The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) |
Source Publication | 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings |
Pages | 11560-11573 |
Conference Date | 20-25 May 2024 |
Conference Place | Torino |
Country | Italy |
Publisher | European Language Resources Association (ELRA) and ICCL |
Abstract | The effective use of monolingual and bilingual knowledge represents a critical challenge within the neural machine translation (NMT) community. In this paper, we propose a modular strategy that facilitates the cooperation of these two types of knowledge in translation tasks, while avoiding the issue of catastrophic forgetting and exhibiting superior model generalization and robustness. Our model is comprised of three functionally independent modules: an encoding module, a decoding module, and a transferring module. The former two acquire large-scale monolingual knowledge via self-supervised learning, while the latter is trained on parallel data and responsible for transferring latent features between the encoding and decoding modules. Extensive experiments in multi-domain translation tasks indicate our model yields remarkable performance, with up to 7 BLEU improvements in out-of-domain tests over the conventional pretrain-and-finetune approach. Our codes are available at https://github.com/NLP2CT/MoNMT. |
Keyword | Catastrophic Forgetting Machine Translation Monolingual And Bilingual Knowledge |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85195948668 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | Faculty of Science and Technology THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU) DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Yang, Baosong; Wong, Derek Fai |
Affiliation | 1.University of Macau, Macao 2.Alibaba Group, China |
First Author Affilication | University of Macau |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Pang, Jianhui,Yang, Baosong,Wong, Derek Fai,et al. MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation[C]:European Language Resources Association (ELRA) and ICCL, 2024, 11560-11573. |
APA | Pang, Jianhui., Yang, Baosong., Wong, Derek Fai., Liu, Dayiheng., Wei, Xiangpeng., Xie, Jun., & Chao, Lidia Sam (2024). MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation. 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, 11560-11573. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment