摘要
【目的】为缓解在公开论坛、企业后台数据仓库的数据自动化分类及存储过程中,由于电商评论文本具有一词多义、信息分布稀疏等特点而造成的处理困难,本文设计基于BERT语言模型和多通道特征抽取的文本分类模型,实现评论文本的自动化分类。【方法】考虑到中文文本的最小表达单元是字,通过BERT的字向量编码联动TextCNN缓解一词多义的问题。为捕获长距离上下文语义信息,模型设计了BERT联动BiLSTM的通道。充分利用BERT的微调机制,使字向量编码根据两个通道的特征抽取方式进行调整,从而得到适配局部和长距离特征抽取的字向量编码结果。模型最终融合两个通道的特征向量,完成文本分类任务。【结果】本文提出的MFFMB(Multi-Features Fusion Model BERT-based)模型在公开的电子商务评论多分类数据集上的准确率高达0.9007,相对于基线模型BERT+TextCNN、BERT+BiLSTM、BERT+LSTM+MaxPooling、BERT+LSTM+Attention分别提升2.36、8.55、4.61、5.11个百分点。同时,实验结果说明BERT和注意力机制的引入,相对于基线模型中的较优者,准确率分别提升约1.48和4.81个百分点。【局限】注意力机制仅在BiLSTM通道引入,没有在全局设计;本文模型未在更多数据集上验证效果。【结论】本文模型能够更好地结合多维度信息,更加充分地挖掘BERT预训练模型在文本分类任务上的作用,提高了分类的准确性。
[Objective] This paper designs a text classification method based on the BERT model and multichannel feature extraction, aiming to accurately conduct automatic classification for e-commence comments. The new model will also address the issues of polysemy and sparse information of comments from public online forums and enterprise data warehouses. [Methods] First, we used BERT’s TextCNN to reduce the polysemy of Chinese words. Then, our model utilized the BERT linkage Bi-LSTM channel to capture the long-distance context semantics. Third, we used BERT’s fine-tuning mechanism to adjust the word vector coding with the extracted features. Finally, the model fused the feature vectors and finished the text classification. [Results] The accuracy of the MFFMB(Multi-Features Fusion Model BERT-based) reached 90.07% on the public data sets of e-commerce comments. Compared with the popular baseline models, the accuracy of the proposed one was improved by 2.36,8.55, 4.61 and 5.11 percentage points. Meanwhile, combining the BERT and attention mechanism improved our models’ accuracy by 1.48 and 4.81 percentage points than their best baseline counterparts. [Limitations] The attention mechanism was only used with the BiLSTM channel. Future research is needed to examine our model with more data sets. [Conclusions] The proposed model could effectively improve the accuracy of text classification.
作者
谢星雨
余本功
Xie Xingyu;Yu Bengong(School of Management,Hefei University of Technology,Hefei 230009,China;Key Laboratory of Process Optimization&Intelligent Decision-making of Ministry of Education,Hefei University of Technology,Hefei 230009,China)
出处
《数据分析与知识发现》
CSSCI
CSCD
北大核心
2022年第1期101-112,共12页
Data Analysis and Knowledge Discovery
基金
国家自然科学基金项目(项目编号:71671057)的研究成果之一。
