UM  > Faculty of Science and Technology
Residential Collegefalse
Status已發表Published
Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages
Tong, Yu1; Guo, Jingzhi2; Zhou, Jizhe2
2022-03-23
Source PublicationIEEE-ACM Transactions on Audio Speech and Language Processing
ISSN2329-9290
Volume30Pages:1521-1530
Abstract

Existing methods consider Word Segmentation (WS) 1 as sequence tagging. Each tag indicates the position of the current 2 character in a segment. The exactness of the position for any 3 non-boundaries character is unnecessary. Any incorrect inner 4 prediction reduces model performance. The position information 5 restricts tag-to-tag transition. Thereby, extra context information 6 and the Conditional Random Field (CRF) network are desired 7 to control unreasonable tag transition. To steer away from the 8 implicit restriction, we propose the Separation(Sp)-Adhesion(Ad), 9 which targets straight on the essential character-to-character 10 connections, to tackle the WS task directly. Merely bigram that 11 is specially tailored for "Sp-Ad" is required and considered as 12 the processing unit to identify the connection states of every two 13 adjacent characters. The elimination of the position restriction 14 makes the model independent of the CRF layer which is widely 15 adopted to revise unreasonable tags. Therefore, CRF can then 16 be substituted with a classification network. We construct the 17 Separation Inference (SpIn) framework based on the bigram 18 features and softmax classification network to tackle the WS 19 task. SpIn significantly reduces the inference complexity, dispels 20 extra context information, and boosts the accuracy of the WS 21 task. Besides its effectiveness in Chinese Word Segmentation, 22 performance boosts on Japanese and Korean Word Segmentation 23 further prove SpIn is universal for East Asian Languages. 24 Moreover, our extensive experiments also verify the cross-domain 25 effectiveness of SpIn by attaining state-of-the-art performances 26 in the benchmark tests of in-domain and cross-domain Chinese Word Segmentation.

Keyword29 Cross-domain Adhesives Complexity Theory East Asian Languages 30 Feature Extraction Optimization Separation Inference Speech Processing Tagging Task Analysis Word Segmentation
DOI10.1109/TASLP.2022.3161142
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaAcoustics ; Engineering
WOS SubjectAcoustics ; Engineering, Electrical & Electronic
WOS IDWOS:000790811400005
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC445 HOES LANE, PISCATAWAY, NJ 08855-4141
Scopus ID2-s2.0-85127035549
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionFaculty of Science and Technology
DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE
Corresponding AuthorGuo, Jingzhi
Affiliation1.Computer and Information Science, University of Macau, 59193 Taipa, Macau, Macao
2.Computer Information Science, University of Macau, 59193 Taipa, Macau, Macao
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Tong, Yu,Guo, Jingzhi,Zhou, Jizhe. Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages[J]. IEEE-ACM Transactions on Audio Speech and Language Processing, 2022, 30, 1521-1530.
APA Tong, Yu., Guo, Jingzhi., & Zhou, Jizhe (2022). Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages. IEEE-ACM Transactions on Audio Speech and Language Processing, 30, 1521-1530.
MLA Tong, Yu,et al."Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages".IEEE-ACM Transactions on Audio Speech and Language Processing 30(2022):1521-1530.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Tong, Yu]'s Articles
[Guo, Jingzhi]'s Articles
[Zhou, Jizhe]'s Articles
Baidu academic
Similar articles in Baidu academic
[Tong, Yu]'s Articles
[Guo, Jingzhi]'s Articles
[Zhou, Jizhe]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Tong, Yu]'s Articles
[Guo, Jingzhi]'s Articles
[Zhou, Jizhe]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.