背景:情感分析是自然语言处理中一项重要但困难的任务。粤语的语言特点,包括它与标准汉语的高度相似性,它的语法和词汇独特性,以及它的口语和多种语言,使其与其他语言不同,并给情绪分析带来额外的挑战。ChatGPT等模型的最新进展提供了潜在的可行解决方案。
目的:本研究调查了GPT-3.5和GPT-4在基于网络的咨询背景下在粤语情感分析中的功效,并将其性能与其他主流方法进行了比较,包括基于词典的方法和机器学习方法。
方法:我们分析了基于网络的转录本,香港的基于文本的咨询服务,包括总共131次个人咨询会议和6169次顾问和寻求帮助者之间的信息。首先,为人类注释开发了一个码本。一个简单的提示(\"这个粤语文本的情绪是肯定的吗,中性,还是负面?只回复情绪标签。\“)然后被给予GPT-3.5和GPT-4来标记每个消息的情绪。GPT-3.5和GPT-4的性能与基于词典的方法和3个最先进的模型进行了比较,包括线性回归,支持向量机,和长短期记忆神经网络。
结果:我们的发现揭示了ChatGPT在情感分类方面的显着准确性,分别使用GPT-3.5和GPT-4,识别阳性的准确率达到92.1%(5682/6169)和95.3%(5880/6169),中性,和负面情绪,从而优于传统的基于词典的方法,准确率为37.2%(2295/6169),和3种机器学习模型,准确率从66%(4072/6169)到70.9%(4374/6169)。
结论:在许多文本分析技术中,ChatGPT表现出卓越的准确性,并成为粤语情绪分析的有前途的工具。这项研究还强调了ChatGPT在现实世界场景中的适用性,例如监测基于文本的咨询服务的质量和检测体内的消息级情绪。这项研究得出的见解为进一步探索ChatGPT在资源不足的语言和心理治疗和自然语言处理等专业领域的能力铺平了道路。
BACKGROUND: Sentiment analysis is a significant yet difficult task in natural language processing. The linguistic peculiarities of
Cantonese, including its high similarity with Standard Chinese, its grammatical and lexical uniqueness, and its colloquialism and multilingualism, make it different from other languages and pose additional challenges to sentiment analysis. Recent advances in models such as ChatGPT offer potential viable solutions.
OBJECTIVE: This study investigated the efficacy of GPT-3.5 and GPT-4 in
Cantonese sentiment analysis in the context of web-based counseling and compared their performance with other mainstream methods, including lexicon-based methods and machine learning approaches.
METHODS: We analyzed transcripts from a web-based, text-based counseling service in Hong Kong, including a total of 131 individual counseling sessions and 6169 messages between counselors and help-seekers. First, a codebook was developed for human annotation. A simple prompt (\"Is the sentiment of this
Cantonese text positive, neutral, or negative? Respond with the sentiment label only.\") was then given to GPT-3.5 and GPT-4 to label each message\'s sentiment. GPT-3.5 and GPT-4\'s performance was compared with a lexicon-based method and 3 state-of-the-art models, including linear regression, support vector machines, and long short-term memory neural networks.
RESULTS: Our findings revealed ChatGPT\'s remarkable accuracy in sentiment classification, with GPT-3.5 and GPT-4, respectively, achieving 92.1% (5682/6169) and 95.3% (5880/6169) accuracy in identifying positive, neutral, and negative sentiment, thereby outperforming the traditional lexicon-based method, which had an accuracy of 37.2% (2295/6169), and the 3 machine learning models, which had accuracies ranging from 66% (4072/6169) to 70.9% (4374/6169).
CONCLUSIONS: Among many text analysis techniques, ChatGPT demonstrates superior accuracy and emerges as a promising tool for
Cantonese sentiment analysis. This study also highlights ChatGPT\'s applicability in real-world scenarios, such as monitoring the quality of text-based counseling services and detecting message-level sentiments in vivo. The insights derived from this study pave the way for further exploration into the capabilities of ChatGPT in the context of underresourced languages and specialized domains like psychotherapy and natural language processing.