Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces.Most conventional methods for emotion recognition using facial expressions use the entire facial image t...Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces.Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model.In contrast,this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions,especially around the eyes,eyebrows,nose,andmouth.Then,we apply a newclassifier using an ensemble network to increase emotion recognition accuracy.The emotion recognition performance was compared with the conventional algorithms using public databases.The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition.In particular,our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds,with an average of 25% higher performance than previous studies.Consequently,the proposed method is expected to recognize facial expressions,especially fear and anger,to help prevent severe accidents by detecting security-related or dangerous actions in advance.展开更多
提出了一种针对眼镜、帽子等头部饰品的单幅图像真实感虚拟试戴技术,其关键在于虚拟饰品的三维注册和虚实图像的合成.首先提出了一种将人脸关键点检测与刚体姿态估计相结合,求解单幅图像中人脸三维注册信息的算法.然后阐述了借助像素颜...提出了一种针对眼镜、帽子等头部饰品的单幅图像真实感虚拟试戴技术,其关键在于虚拟饰品的三维注册和虚实图像的合成.首先提出了一种将人脸关键点检测与刚体姿态估计相结合,求解单幅图像中人脸三维注册信息的算法.然后阐述了借助像素颜色混合和深度缓冲检测技术解决虚实图像合成中遮挡关系和模型材质问题的方法.在AFLW(Annotated Facial Landmarks in the Wild)人脸数据库上对三维注册算法进行了量化测评,结果表明该算法的精度满足虚拟试戴技术的要求.在较大角度姿态变化以及部分遮挡条件下的实验结果表明,提出的虚拟试戴技术快速准确,试戴效果自然逼真.展开更多
We present a novel approach for automatically detecting and tracking facial landmarks acrossposesandexpressionsfromin-the-wild monocular video data,e.g.,You Tube videos and smartphone recordings.Our method does not re...We present a novel approach for automatically detecting and tracking facial landmarks acrossposesandexpressionsfromin-the-wild monocular video data,e.g.,You Tube videos and smartphone recordings.Our method does not require any calibration or manual adjustment for new individual input videos or actors.Firstly,we propose a method of robust 2D facial landmark detection across poses,by combining shape-face canonical-correlation analysis with a global supervised descent method.Since 2D regression-based methods are sensitive to unstable initialization,and the temporal and spatial coherence of videos is ignored,we utilize a coarse-todense 3D facial expression reconstruction method to refine the 2D landmarks.On one side,we employ an in-the-wild method to extract the coarse reconstruction result and its corresponding texture using the detected sparse facial landmarks,followed by robust pose,expression,and identity estimation.On the other side,to obtain dense reconstruction results,we give a face tracking flow method that corrects coarse reconstruction results and tracks weakly textured areas;this is used to iteratively update the coarse face model.Finally,a dense reconstruction result is estimated after it converges.Extensive experiments on a variety of video sequences recorded by ourselves or downloaded from You Tube show the results of facial landmark detection and tracking under various lighting conditions,for various head poses and facial expressions.The overall performance and a comparison with state-of-art methods demonstrate the robustness and effectiveness of our method.展开更多
基金supported by the Healthcare AI Convergence R&D Program through the National IT Industry Promotion Agency of Korea(NIPA)funded by the Ministry of Science and ICT(No.S0102-23-1007)the Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(NRF-2017R1A6A1A03015496).
文摘Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces.Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model.In contrast,this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions,especially around the eyes,eyebrows,nose,andmouth.Then,we apply a newclassifier using an ensemble network to increase emotion recognition accuracy.The emotion recognition performance was compared with the conventional algorithms using public databases.The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition.In particular,our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds,with an average of 25% higher performance than previous studies.Consequently,the proposed method is expected to recognize facial expressions,especially fear and anger,to help prevent severe accidents by detecting security-related or dangerous actions in advance.
文摘提出了一种针对眼镜、帽子等头部饰品的单幅图像真实感虚拟试戴技术,其关键在于虚拟饰品的三维注册和虚实图像的合成.首先提出了一种将人脸关键点检测与刚体姿态估计相结合,求解单幅图像中人脸三维注册信息的算法.然后阐述了借助像素颜色混合和深度缓冲检测技术解决虚实图像合成中遮挡关系和模型材质问题的方法.在AFLW(Annotated Facial Landmarks in the Wild)人脸数据库上对三维注册算法进行了量化测评,结果表明该算法的精度满足虚拟试戴技术的要求.在较大角度姿态变化以及部分遮挡条件下的实验结果表明,提出的虚拟试戴技术快速准确,试戴效果自然逼真.
基金supported by the Harbin Institute of Technology Scholarship Fund 2016the National Centre for Computer Animation,Bournemouth University
文摘We present a novel approach for automatically detecting and tracking facial landmarks acrossposesandexpressionsfromin-the-wild monocular video data,e.g.,You Tube videos and smartphone recordings.Our method does not require any calibration or manual adjustment for new individual input videos or actors.Firstly,we propose a method of robust 2D facial landmark detection across poses,by combining shape-face canonical-correlation analysis with a global supervised descent method.Since 2D regression-based methods are sensitive to unstable initialization,and the temporal and spatial coherence of videos is ignored,we utilize a coarse-todense 3D facial expression reconstruction method to refine the 2D landmarks.On one side,we employ an in-the-wild method to extract the coarse reconstruction result and its corresponding texture using the detected sparse facial landmarks,followed by robust pose,expression,and identity estimation.On the other side,to obtain dense reconstruction results,we give a face tracking flow method that corrects coarse reconstruction results and tracks weakly textured areas;this is used to iteratively update the coarse face model.Finally,a dense reconstruction result is estimated after it converges.Extensive experiments on a variety of video sequences recorded by ourselves or downloaded from You Tube show the results of facial landmark detection and tracking under various lighting conditions,for various head poses and facial expressions.The overall performance and a comparison with state-of-art methods demonstrate the robustness and effectiveness of our method.
