摘要
近年来,基于深度学习的情绪识别技术取得了显著进展。然而,现有方法主要集中在面部图像或视频上,忽略了其他模态信息,导致鲁棒性和稳定性不足。为了解决这一问题,提出了一种融合多模态信息的面部表情识别方法。首先将输入的人脸视频进行远程光电容积脉搏波(Remote Photo plethysmography,rPPG)信号和人脸三维法向量的提取,其次将这两种模态的信息输入其相应的情绪特征提取子网络,提取出对应的情绪特征向量。然后,将这两种模态提取出的情绪特征向量进行融合,生成一个丰富的特征向量,最后将其输入分类器进行情绪分类任务。通过这种多模态信息融合的方式,提高了面部表情识别的准确性和稳定性。对所提方法在不同数据集上进行了验证,实验结果表明,该方法在多样化面部表情识别中的表现优于当前先进的情感识别方法,具有更高的鲁棒性和稳定性。
In recent years,emotion recognition technology based on deep learning has made significant progress.However,existing methods mainly focus on facial images or videos,neglecting other modal information,which leads to insufficient robustness and stability.To address this issue,a facial expression recognition method that integrating multimodal information is proposed.First,remote photo plethysmography(rPPG)signals and 3D facial normal vectors are extracted from the input facial video.Then,the information from these two modalities is fed into their respective emotion feature extraction sub-networks to extract the corresponding emotion feature vectors.After that,the emotion feature vectors extracted from these two modalities are fused to generate a rich feature vector,which is then fed into a classifier for emotion classification tasks.By integrating multimodal information,the accuracy and stability of facial expression recognition are improved.The proposed method has been validated on different datasets,and experimental results show that it outperforms current advanced emotion recognition methods in diverse facial expression recognition tasks,demonstrating higher robustness and stability.
作者
王宇
刘宇昂
赵梦洁
涂晓光
牛知艺
杨明
刘建华
殷举航
朱新宇
石臣鹏
章超
张铖方
WANG Yu;LIU Yuang;ZHAO Mengjie;TU Xiaoguang;NIU Zhiyi;YANG MingLIU;Jianhua;YIN Juhang;ZHU Xinyu;SHI Chenpeng;ZHANG Chao;ZHANG Chengfang(College of Avionics and Electrical Engineering,Civil Aviation Flight University of China,Guanghan 2.Sichuan 618307,China;Province Engineering Technology Research Center of General Aircraft Civil Maintenance,Aviation Flight University of China,Guanghan 3.Affiliated 618307,China;Hospital of Chengdu University of Traditional Chinese Medicine,Chengdu 4.Sichuan 610032,China;Provincial Key Laboratory of Intelligent Policing,Luzhou 5.Department 646000,China;of Road Traffic Management,Sichuan Police College,Luzhou 646000,China)
出处
《电讯技术》
北大核心
2024年第10期1667-1676,共10页
Telecommunication Engineering
基金
中国博士后科学基金(2022M722248)
中央高校基本科研业务费(ZHMH2022-004,J2023-026)
四川省通用航空器维修工程技术研究中心资助课题(GAMRC2023YB06)
智能警务四川省重点实验室开放课题(ZNJW2024KFQN003)。
