Residential Collegefalse
Status已發表Published
A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU
Chen, Guanhua; Yao, Yutong; Wong, Derek F.; Chao, Lidia S.
2024
Conference NameThe 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Source Publication2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings
Pages1778-1788
Conference Date20-05-2024
Conference PlaceTorino
CountryItaly
Abstract

Multi-intent natural language understanding (NLU) presents a formidable challenge due to the model confusion arising from multiple intents within a single utterance. While previous works train the model contrastively to increase the margin between different multi-intent labels, they are less suited to the nuances of multi-intent NLU. They ignore the rich information between the shared intents, which is beneficial to constructing a better embedding space, especially in low-data scenarios. We introduce a two-stage Prediction-Aware Contrastive Learning (PACL) framework for multi-intent NLU to harness this valuable knowledge. Our approach capitalizes on shared intent information by integrating word-level pre-training and prediction-aware contrastive fine-tuning. We construct a pre-training dataset using a word-level data augmentation strategy. Subsequently, our framework dynamically assigns roles to instances during contrastive fine-tuning while introducing a prediction-aware contrastive loss to maximize the impact of contrastive learning. We present experimental results and empirical analysis conducted on three widely used datasets, demonstrating that our method surpasses the performance of three prominent baselines on both low-data and full-data scenarios.

KeywordConversational Systems Natural Language Understanding Prediction-aware Contrastive Learning
URLView the original
Language英語English
Scopus ID2-s2.0-85195913365
Fulltext Access
Citation statistics
Document TypeConference paper
CollectionDEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorWong, Derek F.
AffiliationNLP2CT Lab, Department of Computer and Information Science, University of Macau, Macao
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Chen, Guanhua,Yao, Yutong,Wong, Derek F.,et al. A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU[C], 2024, 1778-1788.
APA Chen, Guanhua., Yao, Yutong., Wong, Derek F.., & Chao, Lidia S. (2024). A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU. 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, 1778-1788.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Chen, Guanhua]'s Articles
[Yao, Yutong]'s Articles
[Wong, Derek F.]'s Articles
Baidu academic
Similar articles in Baidu academic
[Chen, Guanhua]'s Articles
[Yao, Yutong]'s Articles
[Wong, Derek F.]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Chen, Guanhua]'s Articles
[Yao, Yutong]'s Articles
[Wong, Derek F.]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.