摘要
图像情感分析是机器视觉领域热点问题,然而情感判断主观性较强,仅分析完整图像难以准确刻画图像中情感语义,且高质量图像情感数据不足.为此,提出联合多头数据增强与多粒度语义挖掘的图像情感分析模型M2.首先,设计多头数据增强方法,基于自动数据增强与主动样本精选策略构建递进式数据增强模型,从“质”与“量”两个角度提升数据集;其次,引入情感区域检测模型完成情感区域增强,深入挖掘图像中情感语义强烈的局部区域,进而联合局部区域与整幅图像构建多粒度图像;然后,基于深度互学习框架及局部区域完成模型预训练,充分挖掘异构SENet网络之间互补的情感语义,并以迁移学习方式指导多粒度图像情感分析;最后,设计自适应特征融合模块,融合异构SENet特征以完成多粒度语义挖掘,实现图像情感分析.在Twitter I和FI数据集上验证M2模型,其准确率分别达到90.97%和81.14%,优于主流基线.M2拥有泛化性更强的数据增强策略,可以为其训练提供坚实的数据基础,且对应的实证分析效果较好,模型具备一定的实用价值.
Image sentiment analysis is a hot topic in the computer vision field.However,it is hard to accurately characterize the sentiment semantics by only analyzing the whole image since sentiment judgment is usually subjective.And the image samples with high-quality are scarce.To alleviate the two issues,we propose a multi-head data augment and multi-granularity semantics mining(M?)model.First,a progressive data augmentation model is constructed based on automatic data augmentation and active sample refinement.We improve datasets from the perspectives of quality and quantity.Second,an affective region detection model is introduced for sentiment region augmentation.Intense sentiment semantics is deeply mined from these affective local regions.Then we combine local regions with the whole images to create multi-granularity image data.Third,we pretrain the model through the deep mutual learning framework and affective local regions.The complementary sentiment semantics between heterogeneous SENet networks are fully mined,which is transferred in turn to guide the sentiment analysis of multi-granularity images.Finally,an adaptive feature fusion module is proposed to fuse heterogeneous SENet features to complete multi-granularity semantics mining as well as realize image sentiment analysis.The accuracies of M?are 90.97%and 81.14%on TwitterI and FI,respectively,which outperform mainstream baselinesT.he M?contains a data augmentation strategy with powerful generalization ability,which builds a firm data basis for training.Meanwhile,the corresponding empirical analysis is satisfactory,indicating a certain practicality.
作者
张红斌
侯婧怡
石皞炜
吕敬钦
李雄
李广丽
ZHANG Hong-bin;HOU Jing-yi;SHI Hao-wei;LV Jing-qin;LI Xiong;LI Guang-li(School of Software,East China Jiaotong University,Nanchang 330013,China;School of Information Engineering,East China Jiaotong University,Nanchang 330013,China)
出处
《控制与决策》
EI
CSCD
北大核心
2024年第6期2013-2021,共9页
Control and Decision
基金
国家自然科学基金项目(62161011,62361027)
江西省自然科学基金项目(20232BAB202004,20202BABL202044)
江西省主要学科学术和技术带头人培养计划项目(20204BCJL23035)
江西省社会科学基金项目(22TQ01)
江西省重点研发计划重点项目(20223BBE51036)。
关键词
多头数据增强
多粒度语义挖掘
图像情感分析
情感区域检测
深度互学习
SENet
multi-head data augmentation
multi-granularity semantics mining
image sentiment analysis
affective region detection
deep mutual learning
SENet
