UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation
Pang, Jianhui1; Yang, Baosong2; Wong, Derek Fai1; Liu, Dayiheng2; Wei, Xiangpeng2; Xie, Jun2; Chao, Lidia Sam1
2024-05
Conference NameThe 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Source Publication2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings
Pages11560-11573
Conference Date20-25 May 2024
Conference PlaceTorino
CountryItaly
PublisherEuropean Language Resources Association (ELRA) and ICCL
Abstract

The effective use of monolingual and bilingual knowledge represents a critical challenge within the neural machine translation (NMT) community. In this paper, we propose a modular strategy that facilitates the cooperation of these two types of knowledge in translation tasks, while avoiding the issue of catastrophic forgetting and exhibiting superior model generalization and robustness. Our model is comprised of three functionally independent modules: an encoding module, a decoding module, and a transferring module. The former two acquire large-scale monolingual knowledge via self-supervised learning, while the latter is trained on parallel data and responsible for transferring latent features between the encoding and decoding modules. Extensive experiments in multi-domain translation tasks indicate our model yields remarkable performance, with up to 7 BLEU improvements in out-of-domain tests over the conventional pretrain-and-finetune approach. Our codes are available at https://github.com/NLP2CT/MoNMT.

KeywordCatastrophic Forgetting Machine Translation Monolingual And Bilingual Knowledge
URLView the original
Language英語English
Scopus ID2-s2.0-85195948668
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionFaculty of Science and Technology
THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorYang, Baosong; Wong, Derek Fai
Affiliation1.University of Macau, Macao
2.Alibaba Group, China
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Pang, Jianhui,Yang, Baosong,Wong, Derek Fai,et al. MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation[C]:European Language Resources Association (ELRA) and ICCL, 2024, 11560-11573.
APA Pang, Jianhui., Yang, Baosong., Wong, Derek Fai., Liu, Dayiheng., Wei, Xiangpeng., Xie, Jun., & Chao, Lidia Sam (2024). MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation. 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, 11560-11573.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Pang, Jianhui]'s Articles
[Yang, Baosong]'s Articles
[Wong, Derek Fai]'s Articles
Baidu academic
Similar articles in Baidu academic
[Pang, Jianhui]'s Articles
[Yang, Baosong]'s Articles
[Wong, Derek Fai]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Pang, Jianhui]'s Articles
[Yang, Baosong]'s Articles
[Wong, Derek Fai]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.