Residential College | false |
Status | 已發表Published |
Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle | |
Li, Bixuan; Chen, Qifu; Lin, Jinlin; Li, Sai; Yen, Jerome | |
2024 | |
Conference Name | 8th International Conference on Data Mining and Big Data, DMBD 2023 |
Source Publication | Communications in Computer and Information Science |
Volume | 2017 CCIS |
Pages | 165-180 |
Conference Date | 9 December 2023through 12 December 2023 |
Conference Place | Sanya |
Publisher | Springer Science and Business Media Deutschland GmbH |
Abstract | In recent years, the advancement of Artificial Intelligence (AI) technology has brought both convenience and panic. One of the most notable AI systems in recent years was ChatGPT in 2022. In 2023, GPT-4 was released as the latest version. Scholars are increasingly investigating the potential of ChatGPT/GPT-4 for text generation and summarization. Inspired by the principle of “Faithfulness, Expressiveness, and Elegance” in translation, this study investigates the writing and summarizing capabilities of GPT-4, one of the latest AI chatbots. For this purpose, we collected 60 articles from top financial and technology journals, extracted the abstract part, and fed it into GPT-4 to generate abstracts. Three evaluation metrics were created for evaluation: the Text Relevance Score, the AI Detector Score, and the Plagiarism Detector Score. Our findings indicate that abstracts generated by GPT-4 closely resemble the original abstracts without being detected by the plagiarism detector Turnitin in most cases. This implies that GPT-4 can produce logical and reasonable abstracts of articles on its own. Also, we conducted a cross-temporal analysis of GPT-4’s effectiveness and observed continuous and significant improvement. Nevertheless, with the advancement of AI detectors, the abstracts generated by GPT-4 can broadly be recognized as AI-generated. Furthermore, this paper also discusses ethical concerns and future research directions. |
Keyword | Abstract Generation Artificial Intelligence Chatbot Gpt-4 Large Language Models |
DOI | 10.1007/978-981-97-0837-6_12 |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85187708152 |
Fulltext Access | |
Citation statistics | |
Document Type | Conference paper |
Collection | Faculty of Science and Technology DEPARTMENT OF COMPUTER AND INFORMATION SCIENCE |
Affiliation | University of Macau, Taipa, Avenida da Universidade, Macau, China |
First Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Li, Bixuan,Chen, Qifu,Lin, Jinlin,et al. Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle[C]:Springer Science and Business Media Deutschland GmbH, 2024, 165-180. |
APA | Li, Bixuan., Chen, Qifu., Lin, Jinlin., Li, Sai., & Yen, Jerome (2024). Assessing GPT-4 Generated Abstracts: Text Relevance and Detectors Based on Faithfulness, Expressiveness, and Elegance Principle. Communications in Computer and Information Science, 2017 CCIS, 165-180. |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment