Residential College | false |
Status | 已發表Published |
Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation | |
Zhexuan Wang1; Shudong Liu2; Xuebo Liu1; Miao Zhang1; Derek F. Wong2; Min Zhang1 | |
2024-08 | |
Conference Name | The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) |
Source Publication | Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) |
Pages | 9458-9469 |
Conference Date | 11-16 August 2024 |
Conference Place | Bangkok |
Country | Thailand |
Publisher | Association for Computational Linguistics (ACL) |
Abstract | kNN-MT has utilized neighborhood knowledge for auxiliary decoding, significantly improving translation performance. Subsequently, kNNKD transitions the use of neighborhood knowledge from the decoding phase to the training phase, to address the temporal and spatial inefficiencies inherent in kNN-MT. However, kNN-KD transfers all the kNN knowledge arbitrarily, which has the potential to restrict the learning of student models. In this paper, we propose a novel domain-aware kNN-KD method, which filters out domain-relevant neighborhood knowledge for learning in the distillation process. Notably, this entire process exclusively utilizes the neighborhood knowledge of the original model, eliminating the need for establishing any additional datastores. Experiments on four domain translation tasks demonstrate that our method achieves state-of-the-art performance, realizing an average gain of 1.55 COMET and 1.42 BLEU scores, by further enhancing the translation of rare words. Source code can be accessed at https://github.com/wangzx1219/Dk-KD. |
DOI | 10.18653/v1/2024.findings-acl.563 |
Language | 英語English |
Scopus ID | 2-s2.0-85205284174 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Xuebo Liu |
Affiliation | 1.Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China 2.NLP2CT Lab, Department of Computer and Information Science, University of Macau |
Recommended Citation GB/T 7714 | Zhexuan Wang,Shudong Liu,Xuebo Liu,et al. Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation[C]:Association for Computational Linguistics (ACL), 2024, 9458-9469. |
APA | Zhexuan Wang., Shudong Liu., Xuebo Liu., Miao Zhang., Derek F. Wong., & Min Zhang (2024). Domain-Aware k-Nearest-Neighbor Knowledge Distillation for Machine Translation. Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 9458-9469. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment