摘要
在文本分类任务中,双向门控循环单元(Bidirectional Gated Recurrent Unit,BiGRU)被广泛使用,其不仅能提取文本上下文语义信息和长距离依赖关系,还可以避免出现传统RNN中存在的梯度弥散或爆炸问题.然而,BiGRU在捕获文本局部特征方面存在不足.本文提出一种基于自注意力和双向门控循环单元的文本分类模型(Self-attention and Bidirectional-gated-recurrent Unit based Text Classification,SBUTC),利用自注意力机制关注对分类贡献较大的文本部分,使用含有不同尺寸卷积核的多通道CNN提取不同粒度的文本局部特征;通过含有跳层连接结构的堆叠BiGRU网络提取文本间上下文语义信息和长距离依赖关系;将CNN和BiGRU的输出进行特征融合,训练分类器对不同类型的文本信息进行分类.在ChnSentiCorp数据集和THUCNews_Title数据集上的对比实验结果表明,本文提出的模型在分类准确率和F1值上优于其他对比模型.
In text classification tasks,the Bidirectional Gated Recurrent Unit(BiGRU)network is widely used.It can not only solve the problem of gradient disappearance or gradient explosion in traditional RNN model,but also extract the contextual semantic information and long-distance dependency of the text.However,it cannot capture the local features of the text.This paper proposes a SBUTC(Self-attention and Bidirectional-gated-recurrent Unit based Text Classification)model.Firstly,the self-attention mechanism is used to focus on the parts of text that contribute a lot to classification.Then,the Multi-channel CNN with convolution kernels of different sizes can effectively extract local feature information of text with different granularities.At the same time,the contextual semantic information and long-distance dependency are extracted by the stacked BiGRU network with residual connection structure.Finally,the outputs of CNN and BiGRU are merged to train the classifier to classify the text information.The experimental results on the ChnSentiCorp dataset and THUCNews_Title dataset show that compared to other models or algorithms,the proposed model can achieve better classification accuracy and F1-Score.
作者
石磊
王明宇
宋哲理
陶永才
卫琳
高宇飞
范雨欣
SHI Lei;WANG Ming-yu;SONG Zhe-li;TAO Yong-cai;WEI Lin;GAO Yu-fei;FAN Yu-xin(School of Information Engineering,Zhengzhou University,Zhengzhou 450001,China;School of Software,Zhengzhou University,Zhengzhou 450002,China;Department of Information Technology,Zhengzhou Vocational College of Finance and Taxation,Zhengzhou 454048,China)
出处
《小型微型计算机系统》
CSCD
北大核心
2022年第12期2541-2548,共8页
Journal of Chinese Computer Systems
基金
国家重点研发计划项目(2018YFB1701401,2020YFB1712401-1)资助
国家自然科学基金青年科学基金项目(62006210)资助
2020年度河南省重大公益专项项目(201300210500)资助
河南省高等学校重点科研项目(21B520018)资助
铁道警察学院基科项目(2020TJJBKY002)资助。
关键词
文本分类
自注意力机制
双向门控循环单元
卷积神经网络
text classification
self-attention mechanism
BiGRU(Bidirectional Gated Recurrent Unit)
CNN(convolutional neural network)
