摘要
自然图像情感分类在分析用户需求、监控网络舆情等方面具有重要意义。然而基于深度学习的分类算法存在训练过程难以控制、分类结果缺乏解释的问题。为此提出一种人类知识驱动的深度学习结构优化算法。首先通过特征可视化显示卷积神经网络提取的情感特征;其次结合人类对图像情感可视化结果的感知来优化网络结构,利用人类知识驱动网络,重点学习情感信息更明显的特征;最后对所构建网络的参数进行微调,使其更适用于自然图像情感分类任务。在Twitter情感图像数据集上与其他分类方法的对比实验表明,所提出的算法获得了88.1%的分类准确率,优于其他方法。消融实验证明网络优化结果比未优化提高了8.1%。类激活图、空间位置和神经元组特征可视化直观解释了模型运作的过程与原因,进一步证实算法识别自然图像情感的能力。
Automatically analyzing the sentiment of natural images plays a vital role in analyzing user needs and network public opinion monitoring.However,the training processes of deep learning-based classification algorithms are too difficult to be controlled,and their classification results are always lack of interpretation.A deep learning structure optimization algorithm with human cognition was proposed to classify image sentiment.Firstly,the emotional features extracted were visualized by the convolutional neural networks.Then,the network structure was optimized by combining with human’s subjective perception of image emotion,and the network structure was driven by human knowledge to focus on the apparent features of emotional information.Finally,the parameters of the rebuilt network were fine-tuned to make it more suitable for images sentiment classification task.Contrastive experiments on the Twitter dataset systematically demonstrate that the proposed algorithm achieves 88.1%classification accuracy,which has superior performance than other methods.Ablation experiments confirm that our network optimization improves the classification effect by 8.1%.Besides,the process and reason were intuitively explained for the model operation through class activation maps,spatial location visualization and neuron group visualization.The visualization experimental results further demonstrate the ability of proposed algorithm to recognize the sentiment of natural images.
作者
盛家川
陈雅琦
王君
韩亚洪
Sheng Jiachuan;Chen Yaqi;Wang Jun;Han Yahong(School of Science and Technology,Tianjin University of Finance&Economics,Tianjin 300222,China;Laboratory of Fintech and Risk Management,Tianjin 300222,China;School of Management Science and Engineering,Tianjin University of Finance&Economics,Tianjin 300222,China;College of Intelligence and Computing,Tianjin University,Tianjin 300072,China)
出处
《红外与激光工程》
EI
CSCD
北大核心
2020年第11期256-265,共10页
Infrared and Laser Engineering
基金
国家自然科学基金(61502331,61876130)
教育部人文社科项目(18YJA630057,19YJA630046)
天津市自然科学基金(18JCYBJC85100)
天津市企业科技特派员项目(19JCTPJC56300)。
关键词
图像情感
图像分类
卷积神经网络
可视化
类激活图
image sentiment
image classification
convolutional neural networks
visualization
class activation map
