Text sentiment classification approach with integrated local and global prominent features

doi:10.3969/j.issn.1000-1565.2021.02.014

Abstract

Abstract: The model combining convolutional neural network(CNN)with Bi-directional long short-term memory(BiLSTM)feature(CNN_BiLSTM)is popular for sentiment analysis. It takes into account the local and global features to realize sentiment classification of the text. However, it ignores the importance of the features for the classification results, and does not make full use of features between words, which result in low classification efficiency and accuracy. Thus, a model based on a CNN that integrates multiple convolution kernels and a bidirectional long-term and short-term memory network of attention(MCNN_Att-BiLSTM)for- DOI:10.3969/j.issn.1000-1565.2021.02.014集成局部和全局关键特征的文本情感分类方法柴变芳¹,杨蕾¹,王建岭²,李仁玲²(1. 河北地质大学信息工程学院,河北石家庄 050031;2. 河北中医学院图书馆,河北石家庄 050200)摘要:融合卷积神经网络(convolutional neural network,CNN)和双向长短期记忆网络(Bi-directional long short-term memory,BiLSTM)的情感分析模型(CNN_BiLSTM)是一个流行的模型,其学习文本的局部特征和全局特征实现情感分类,但是忽略了特征对分类结果的重要程度,且没充分利用词语间的特征,导致分类准确率不高.提出一种集成基于多卷积核的卷积神经网络和注意力双向长短期记忆网络特征的文本情感分类方法(MCNN_Att-BiLSTM),其集成局部和全局的重要特征作为文本语义特征,该特征进而用于训练文本情感分类器XGBoost(eXtreme gradient Boosting).该方法基于注意力机制的BiLSTM提取对分类影响大的全局关键特征,基于多卷积核的CNN获得更全面的词语间特征,为集成分类器准备了有效分类的特征.实验结果表明,该模型具有更好的情感分类准确率,与CNN_BiLSTM模型相比,在IMDB数据集上准确率提升了1.75%,在txt-sentoken数据集上准确率提升了1.67%,在谭松波-酒店评论数据集上准确率提升了3.81%.关键词:情感分析;CNN;BiLSTM;XGBoost;特征融合中图分类号:TP391 文献标志码:A 文章编号:1000-1565(2021)02-0201-11Text sentiment classification approach with integrated local and global prominent featuresCHAI Bianfang¹, YANG Lei¹,WANG Jianling²,LI Renling²(1. College of Information Engineering, Hebei GEO University, Shijiazhuang 050031, China;2. Library, Hebei University of Chinese Medicine, Shijiazhuang 050200, China)Abstract: The model combining convolutional neural network(CNN)with Bi-directional long short-term memory(BiLSTM)feature(CNN_BiLSTM)is popular for sentiment analysis. It takes into account the local and global features to realize sentiment classification of the text. However, it ignores the importance of the features for the classification results, and does not make full use of features between words, which result in low classification efficiency and accuracy. Thus, a model based on a CNN that integrates multiple convolution kernels and a bidirectional long-term and short-term memory network of attention(MCNN_Att-BiLSTM)for- 收稿日期:2020-03-12 基金项目:国家自然科学基金资助项目(81473773);河北省自然科学基金资助项目(F2019403070);河北省教育厅重点项目(ZD2020175) 第一作者:柴变芳(1979—),女,山西运城人,河北地质大学教授,博士,主要从事机器学习、复杂网络分析研究.E-mail:chaibianfang@163.com 通信作者:李仁玲(1973—),女,河北巨鹿人,河北中医学院副研究馆员,主要从事数据检索和大数据分析研究. E-mail:w3365@126.com王建岭(1973—),男,河北巨鹿人,河北中医学院教授,主要从事网络安全和大数据挖掘研究.E-mail:wang_jl@126.com第2期柴变芳等:集成局部和全局关键特征的文本情感分类方法text sentiment classification is proposed. It integrates local and global prominent features as semantic features of the text, which are used as inputs to train a XGBoost(eXtreme gradient Boosting)classifier to realize text sentiment classification. It utilizes the attention mechanism based on the BiLSTM to fully captain the global prominent features that affect the classification results largely. In addition, it utilizes the CNN with multi-convolution kernel to obtain more comprehensive inter-word features. Experimental results show that the model is better compared to the CNN_BiLSTM model, which improves the accuracy rate by 1.75% on the IMDB dataset, and 1.67% on the txt-sentoken dataset, and 3.81% on the Tan Songbo-hotel review dataset.

Key words: sentiment analysis, CNN, BiLSTM, XGBoost, feature fusion

CLC Number:

TP391

CHAI Bianfang, YANG Lei,WANG Jianling,LI Renling. Text sentiment classification approach with integrated local and global prominent features[J]. Journal of Hebei University(Natural Science Edition), 2021, 41(2): 201-211.

References

[1] 李然,林政,林海伦,等.文本情绪分析综述[J].计算机研究与发展,2018,55(1): 30-52. DOI:10.7544/issn1000-1239.2018.20170055.
[2] NASUKAWA T, YI J. Sentiment analysis: capturing favorability using natural language processing[Z].The international Conference on Knowledge Capture, Sanibel Island,2003. DOI:10.1145/945645.945658.
[3] 韩彤晖,杨东强,马宏伟.一种利用情感词统计信息构造文本特征表示的方法[J].计算机应用研究,2018,36(7): 2087-2092. DOI:10.19734/j.issn.1001-3695.2018.01.0035.
[4] CHEN L, YANG Y C. Emotional speaker recognition based on i-vector through Atom Aligned Sparse Representation[Z].IEEE International Conference on Acoustics, Speech and Signal Processing,Vancouver,BC, Canada. DOI:10.1109/icassp.2013.6639174.
[5] 阳庆玲,郑志伟,邱佳玲,等.基于表情符号的文本情感分析研究[J].现代预防医学,2019,46(9): 1537-1540.
[6] RATHI M, MALIK A, VARSHNEY D, et al. Sentiment analysis of tweets using machine learning approach[Z].Eleventh International Conference on Contemporary Computing(IC3), Noida,2018. DOI:10.1109/ic3.2018.8530517.
[7] LE H T, CERISARA C, DENIS A. Do convolutional networks need to be deep for text classification[Z].The AAAI Conference on Artificial Intelligence,New Orleans,Louisiana,2018.
[8] 陈珂,梁斌,柯文德,等.基于多通道卷积神经网络的中文微博情感分析[J].计算机研究与发展,2018,55(5): 945-957. DOI:10.7544/issn1000-1239.2018.20170049.
[9] IRSOY O, CARDIE C. Deep recursive neural networks for compositionality in language[J]. Advances in neural information processing systems,2014,3:2096-2014.DOI:10.5555/2969.2969061.
[10] 李洋,董红斌.基于CNN和BiLSTM网络特征融合的文本情感分析[J].计算机应用,2018,38(11): 3075-3080. DOI:10.11772/j.issn.1001-9081.2018041289.
[11] ARAQUE O, CORCUERA-PLATAS I, SÁNCHEZ-RADA J F, et al. Enhancing deep learning sentiment analysis with ensemble techniques in social applications[J]. Expert Syst Appl, 2017, 77: 236-246. DOI:10.1016/j.eswa.2017.02.002.
[12] 苏兵杰,周亦鹏,梁勋鸽.基于XGBoost算法的电商评论文本情感识别模型[J].物联网技术,2018,8(1): 54-57. DOI:10.16667/j.issn.2095-1302.2018.01.015.
[13] 龚维印,王力.基于卷积神经网络和XGBoost的文本分类[J].通信技术,2018,51(10): 2337-2342. DOI:10.3969/j.issn.1002-0802.2018.10.012.
[14] OKADA S, OHZEKI M, TAGUCHI S. Efficient partition of integer optimization problems with one-hot encoding[J]. Sci Rep, 2019, 9(1): 1-12. DOI:10.1038/s41598-019-49539-6.
[15] 於雯,周武能.基于LSTM的商品评论情感分析[J].计算机系统应用,2018,27(8): 159-163. DOI:10.15888/j.cnki.csa.006483.
[16] ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification[Z]. The 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016. DOI:10.18653/v1/p16-2034.
[17] CHEN T Q, GUESTRIN C. XGBoost: a scalable tree boosting system[Z]. The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco,2016. DOI:10.1145/2939672.2939785.
[18] CHEN T Q, GUESTRIN C. XGBoost: a scalable tree boosting system[Z]. The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, 2016. DOI:10.1145/2939672.2939785.
[19] PANG B, LEE L. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts[Z].The 42nd Annual Meeting on Association for Computational Linguistics, Barcelona, Spain,2004. DOI:10.3115/1218955.1218990.
[20] 蔡林森,彭超,陈思远,等.基于多样化特征卷积神经网络的情感分析[J].计算机工程,2019,45(4): 169-174,180. DOI:10.19678/j.issn.1000-3428.0050338. (