摘要
随着算力的提升,文本分类算法已进入深度学习时代。文章以深度学习下的自适应微调长文本分类模型为基础,针对其策略网络存在决策能力不足与离散噪声这一问题,结合现有分层模型展开研究,提出融合层编码的层级自适应微调长文本分类模型,力求推进模型在长文本分类任务上的性能。首先,文章重构策略网络,将策略网络迁移至模型内部,消除离散噪声,提高决策精度。其次,考虑预训练模型的层级特征差异,文章提出层编码,为策略网络提供层位置信息,提高策略网络对特征的层位置感知。文章基于Yelp-2013、IMDB、Reuters 3个国际数据集,利用对比实验、烧蚀实验验证模型性能。实验表明,文章提出的长文本分类模型相较于基线模型在3个数据集上的性能更优。
With the improvement of computational power,text classification algorithms have entered the era of deep learning.This article is based on an adaptive fine-tuning long text classification model under deep learning and focuses on the issues of insufficient decision-making ability and discrete noise in its policy network.By combining existing hierarchical models,a hierarchical adaptive fine-tuning long text classification model with fusion layer encoding is proposed,aiming to advance the performance of the model in long text classification tasks.Firstly,this article reconstructs the policy network by transferring it internally within the model,eliminating discrete noise and improving decision accuracy.Secondly,considering the hierarchical feature differences of pre-trained models,this article introduces layer encoding to provide layer position information to the policy network,enhancing the decision network’s perception of feature layer positions.Based on the Yelp-2013,IMDB,and Reuters international datasets,this article validates the model’s performance through comparative experiments and ablative experiments.The results demonstrate that the proposed long text classification model outperforms the baseline model on all three datasets.
作者
郑坚
王俊鑫
陈奕林
林灵鑫
侯子豪
Zheng Jianyi;Wang Junxin;Chen Yilin;Lin Lingxin;Hou Zihao(Guangdong University of Technology,Guangzhou 510000,China)
出处
《无线互联科技》
2023年第18期139-142,共4页
Wireless Internet Technology
关键词
长文本分类
预训练模型
注意力机制
循环神经网络
long text classification
pre-training model
attention mechanism
recurrent neural netw
