摘要
针对短文本缺乏足够共现信息所产生的词与词之间弱连接,且难以获取主题词的情况,导致面向短文本分类工作需要人工标注大量的训练样本,以及产生特征稀疏和维度爆炸的问题,提出了一种基于注意力机制和标签图的单词共生短文本分类模型(WGA-BERT)。首先利用预先训练好的BERT模型计算上下文感知的文本表示,并使用WNTM对每个单词的潜在单词组分布进行建模,以获取主题扩展特征向量;其次提出了一种标签图构造方法捕获主题词的结构和相关性;最后,提出了一种注意力机制建立主题词之间,以及主题词和文本之间的联系,解决了数据稀疏性和主题文本异构性的问题。实验结果表明,WGA-BERT模型对于新闻评论类的短文本分类,比传统的机器学习模型在分类精度上平均提高了3%。
Due to the lack of sufficient co-occurrence information in short text, weak connections between words, and it is difficult to obtain subject words, which leads to the need to manually label a large number of training samples for short text classification, and the problems of sparse features and dimension explosion.This paper proposed a word symbiotic short text classification model based on attention mechanism and label graph(WGA-BERT).Firstly, this paper used the pretrained BERT model to calculate the context aware text representation, and used WNTM to model the potential word group distribution of each word to obtain the topic expansion feature vector.Secondly, this paper used a tag graph construction method to capture the structure and relevance of subject words.Finally, this paper used an attention mechanism to establish the relationship between subject words and between subject words and text, which solved the problems of data sparsity and subject text heterogeneity.The experimental results show that the WGA-BERT model improves the classification accuracy by an average of 3% compared with the traditional machine learning model.
作者
杨阳
刘恩博
顾春华
裴颂文
Yang Yang;Liu Enbo;Gu Chunhua;Pei Songwen(School of Optical-Electrical&Computer Engineering,University of Shanghai for Science&Technology,Shanghai 200082,China)
出处
《计算机应用研究》
CSCD
北大核心
2022年第3期711-715,750,共6页
Application Research of Computers
基金
国家自然科学基金资助项目(61975124)
上海自然科学基金资助项目(20ZR1438500)
上海市科委科技行动计划资助项目(20DZ2308700)
上海市经信委软件和集成电路产业发展专项(RX-RJJC-02-20-4212)。
关键词
短文本分类
词嵌入
单词网络主题模型
注意力机制
short text classification
word embedding
word network topic model(WNTM)
attention mechanism
