Residential College | false |
Status | 已發表Published |
Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages | |
Tong, Yu1; Guo, Jingzhi2; Zhou, Jizhe2 | |
2022-03-23 | |
Source Publication | IEEE-ACM Transactions on Audio Speech and Language Processing |
ISSN | 2329-9290 |
Volume | 30Pages:1521-1530 |
Abstract | Existing methods consider Word Segmentation (WS) 1 as sequence tagging. Each tag indicates the position of the current 2 character in a segment. The exactness of the position for any 3 non-boundaries character is unnecessary. Any incorrect inner 4 prediction reduces model performance. The position information 5 restricts tag-to-tag transition. Thereby, extra context information 6 and the Conditional Random Field (CRF) network are desired 7 to control unreasonable tag transition. To steer away from the 8 implicit restriction, we propose the Separation(Sp)-Adhesion(Ad), 9 which targets straight on the essential character-to-character 10 connections, to tackle the WS task directly. Merely bigram that 11 is specially tailored for "Sp-Ad" is required and considered as 12 the processing unit to identify the connection states of every two 13 adjacent characters. The elimination of the position restriction 14 makes the model independent of the CRF layer which is widely 15 adopted to revise unreasonable tags. Therefore, CRF can then 16 be substituted with a classification network. We construct the 17 Separation Inference (SpIn) framework based on the bigram 18 features and softmax classification network to tackle the WS 19 task. SpIn significantly reduces the inference complexity, dispels 20 extra context information, and boosts the accuracy of the WS 21 task. Besides its effectiveness in Chinese Word Segmentation, 22 performance boosts on Japanese and Korean Word Segmentation 23 further prove SpIn is universal for East Asian Languages. 24 Moreover, our extensive experiments also verify the cross-domain 25 effectiveness of SpIn by attaining state-of-the-art performances 26 in the benchmark tests of in-domain and cross-domain Chinese Word Segmentation. |
Keyword | 29 Cross-domain Adhesives Complexity Theory East Asian Languages 30 Feature Extraction Optimization Separation Inference Speech Processing Tagging Task Analysis Word Segmentation |
DOI | 10.1109/TASLP.2022.3161142 |
URL | View the original |
Indexed By | SCIE |
Language | 英語English |
WOS Research Area | Acoustics ; Engineering |
WOS Subject | Acoustics ; Engineering, Electrical & Electronic |
WOS ID | WOS:000790811400005 |
Publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC445 HOES LANE, PISCATAWAY, NJ 08855-4141 |
Scopus ID | 2-s2.0-85127035549 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Corresponding Author | Guo, Jingzhi |
Affiliation | 1.Computer and Information Science, University of Macau, 59193 Taipa, Macau, Macao 2.Computer Information Science, University of Macau, 59193 Taipa, Macau, Macao |
First Author Affilication | University of Macau |
Corresponding Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Tong, Yu,Guo, Jingzhi,Zhou, Jizhe. Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages[J]. IEEE-ACM Transactions on Audio Speech and Language Processing, 2022, 30, 1521-1530. |
APA | Tong, Yu., Guo, Jingzhi., & Zhou, Jizhe (2022). Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages. IEEE-ACM Transactions on Audio Speech and Language Processing, 30, 1521-1530. |
MLA | Tong, Yu,et al."Separation Inference: A Unified Framework for Word Segmentation in East Asian Languages".IEEE-ACM Transactions on Audio Speech and Language Processing 30(2022):1521-1530. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment