UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation
Zhexuan Wang1; Shudong Liu2; Xuebo Liu1; Miao Zhang1; Derek F. Wong2; Min Zhang1
2024-08
Conference NameThe 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
Source PublicationFindings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024)
Pages9458-9469
Conference Date11-16 August 2024
Conference PlaceBangkok
CountryThailand
PublisherAssociation for Computational Linguistics (ACL)
Abstract

kNN-MT has utilized neighborhood knowledge for auxiliary decoding, significantly improving translation performance. Subsequently, kNNKD transitions the use of neighborhood knowledge from the decoding phase to the training phase, to address the temporal and spatial inefficiencies inherent in kNN-MT. However, kNN-KD transfers all the kNN knowledge arbitrarily, which has the potential to restrict the learning of student models. In this paper, we propose a novel domain-aware kNN-KD method, which filters out domain-relevant neighborhood knowledge for learning in the distillation process. Notably, this entire process exclusively utilizes the neighborhood knowledge of the original model, eliminating the need for establishing any additional datastores. Experiments on four domain translation tasks demonstrate that our method achieves state-of-the-art performance, realizing an average gain of 1.55 COMET and 1.42 BLEU scores, by further enhancing the translation of rare words. Source code can be accessed at https://github.com/wangzx1219/Dk-KD.

DOI10.18653/v1/2024.findings-acl.563
Language英語English
Scopus ID2-s2.0-85205284174
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorXuebo Liu
Affiliation1.Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China
2.NLP2CT Lab, Department of Computer and Information Science, University of Macau
Recommended Citation
GB/T 7714
Zhexuan Wang,Shudong Liu,Xuebo Liu,et al. Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation[C]:Association for Computational Linguistics (ACL), 2024, 9458-9469.
APA Zhexuan Wang., Shudong Liu., Xuebo Liu., Miao Zhang., Derek F. Wong., & Min Zhang (2024). Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation. Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 9458-9469.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Zhexuan Wang]'s Articles
[Shudong Liu]'s Articles
[Xuebo Liu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Zhexuan Wang]'s Articles
[Shudong Liu]'s Articles
[Xuebo Liu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Zhexuan Wang]'s Articles
[Shudong Liu]'s Articles
[Xuebo Liu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.