A virtual cosmetics try-on system provides a realistic try-on experience for consumers and helps them efficiently choose suitable cosmetics.In this article,we propose a real-time augmented reality virtual cosmetics tr...A virtual cosmetics try-on system provides a realistic try-on experience for consumers and helps them efficiently choose suitable cosmetics.In this article,we propose a real-time augmented reality virtual cosmetics try-on system for smartphones(ARCosmetics),taking speed,accuracy,and stability into consideration at each step to ensure a better user experience.A novel and very fast face tracking method utilizes the face detection box and the average position of facial landmarks to estimate the faces in continuous frames.A dynamic weight Wing loss is introduced to assign a dynamic weight to every landmark by the estimated error during training.It balances the attention between small,medium,and large range error and thus increases the accuracy and robustness.We also designed a weighted average method to utilize the information of the adjacent frame for landmark refinement,guaranteeing the stability of the generated landmarks.Extensive experiments conducted on a large 106-point facial landmark dataset and the 300-VW dataset demonstrate the superior performance of the proposed method compared to other state-of-the-art methods.We also conducted user satisfaction studies further to verify the efficiency and effectiveness of our ARCosmetics system.展开更多
This paper proposes a novel nonlinear correlation filter for facial landmark localization. Firstly, we prove that SVM as a classifier can also be used for localization. Then, soft constrained Minimum Average Correlati...This paper proposes a novel nonlinear correlation filter for facial landmark localization. Firstly, we prove that SVM as a classifier can also be used for localization. Then, soft constrained Minimum Average Correlation Energy filter （soft constrained MACE） is proposed, which is more resistent to overfittings to training set than other variants of correlation filter. In order to improve the performance for the multi-mode of the targets, locally linear framework is introduced to our model, which results in Fourier Locally Linear Soft Constraint MACE （FL^2 SC-MACE）. Furthermore, we formulate the fast implementation and show that the time consumption in test process is independent of the number of training samples. The merits of our method include accurate localization performance, desiring generalization capability to the variance of objects, fast testing speed and insensitivity to parameter settings. We conduct the cross-set eye localization experiments on challenging FRGC, FERET and BioID datasets. Our method surpasses the state-of-arts especially in pixelwise accuracy.展开更多
As a typical biometric cue with great diversities, smile is a fairly influential signal in social interaction, which reveals the emotional feeling and inner state of a person. Spontaneous and posed smiles initiated by...As a typical biometric cue with great diversities, smile is a fairly influential signal in social interaction, which reveals the emotional feeling and inner state of a person. Spontaneous and posed smiles initiated by different brain systems have differences in both morphology and dynamics. Distinguishing the two types of smiles remains challenging as discriminative subtle changes need to be captured, which are also uneasily observed by human eyes. Most previous related works about spontaneous versus posed smile recognition concentrate on extracting geometric features while appearance features are not fully used, leading to the loss of texture information. In this paper, we propose a region-specific texture descriptor to represent local pattern changes of different facial regions and compensate for limitations of geometric features. The temporal phase of each facial region is divided by calculating the intensity of the corresponding facial region rather than the intensity of only the mouth region. A mid-level fusion strategy of support vector machine is employed to combine the two feature types. Experimental results show that both our proposed appearance representation and its combination with geometry-based facial dynamics achieve favorable performances on four baseline databases： BBC, SPOS, MMI, and UvA-NEMO.展开更多
三维局部约束模型(3D Constrained Local Model,CLM-Z)算法,综合利用灰度和深度信息检测三维人脸数据中的特征点(如眼角、鼻尖和嘴角),实现了较高的检测精度。CLM-Z方法一般使用人脸位置和平均三维人脸模型进行初始化。设计了四个实验...三维局部约束模型(3D Constrained Local Model,CLM-Z)算法,综合利用灰度和深度信息检测三维人脸数据中的特征点(如眼角、鼻尖和嘴角),实现了较高的检测精度。CLM-Z方法一般使用人脸位置和平均三维人脸模型进行初始化。设计了四个实验定量地分析CLM-Z参数初始化对算法精度的影响:在BU-4DFE库上评估CLM-Z算法精度;通过平移人脸边界框扰动平移参数的初始值;通过缩放人脸边界框扰动尺度参数的初始值;通过给定绕y轴和z轴的旋转角扰动旋转参数的初始值。实验结果表明,CLM-Z算法可容忍平移扰动约为人脸宽的1/6,在(0.75,1.50)缩放范围内算法精度不会下降,可容忍y轴和z轴旋转角约20°。基于以上评估结果,进一步提出在纹理图像上检测特征点作为初始化,然后再进行CLM-Z迭代。在BU-4DFE数据库上的评估结果证明,该初始化方法能有效提升CLM-Z方法的特征点定位精度。展开更多
基金supported in part by the National Key R&D Program of China(2021ZD0140407)in part by the National Natural Science Foundation of China(Grant No.U21A20523).
文摘A virtual cosmetics try-on system provides a realistic try-on experience for consumers and helps them efficiently choose suitable cosmetics.In this article,we propose a real-time augmented reality virtual cosmetics try-on system for smartphones(ARCosmetics),taking speed,accuracy,and stability into consideration at each step to ensure a better user experience.A novel and very fast face tracking method utilizes the face detection box and the average position of facial landmarks to estimate the faces in continuous frames.A dynamic weight Wing loss is introduced to assign a dynamic weight to every landmark by the estimated error during training.It balances the attention between small,medium,and large range error and thus increases the accuracy and robustness.We also designed a weighted average method to utilize the information of the adjacent frame for landmark refinement,guaranteeing the stability of the generated landmarks.Extensive experiments conducted on a large 106-point facial landmark dataset and the 300-VW dataset demonstrate the superior performance of the proposed method compared to other state-of-the-art methods.We also conducted user satisfaction studies further to verify the efficiency and effectiveness of our ARCosmetics system.
文摘This paper proposes a novel nonlinear correlation filter for facial landmark localization. Firstly, we prove that SVM as a classifier can also be used for localization. Then, soft constrained Minimum Average Correlation Energy filter （soft constrained MACE） is proposed, which is more resistent to overfittings to training set than other variants of correlation filter. In order to improve the performance for the multi-mode of the targets, locally linear framework is introduced to our model, which results in Fourier Locally Linear Soft Constraint MACE （FL^2 SC-MACE）. Furthermore, we formulate the fast implementation and show that the time consumption in test process is independent of the number of training samples. The merits of our method include accurate localization performance, desiring generalization capability to the variance of objects, fast testing speed and insensitivity to parameter settings. We conduct the cross-set eye localization experiments on challenging FRGC, FERET and BioID datasets. Our method surpasses the state-of-arts especially in pixelwise accuracy.
基金the National Natural Science Foundation of China （No. 60675025）, the National High-Tech R＆D Program （863） of China （No. 2006AA04Z247）, the Scientific and Tech- nical Innovation Commission of Shenzhen Municipality, China （Nos. JCYJ20130331144631730 and JCYJ20130331144716089）, and the Specialized Research Fund for the Doctoral Program of Higher Education, China （No. 20130001110011）
文摘As a typical biometric cue with great diversities, smile is a fairly influential signal in social interaction, which reveals the emotional feeling and inner state of a person. Spontaneous and posed smiles initiated by different brain systems have differences in both morphology and dynamics. Distinguishing the two types of smiles remains challenging as discriminative subtle changes need to be captured, which are also uneasily observed by human eyes. Most previous related works about spontaneous versus posed smile recognition concentrate on extracting geometric features while appearance features are not fully used, leading to the loss of texture information. In this paper, we propose a region-specific texture descriptor to represent local pattern changes of different facial regions and compensate for limitations of geometric features. The temporal phase of each facial region is divided by calculating the intensity of the corresponding facial region rather than the intensity of only the mouth region. A mid-level fusion strategy of support vector machine is employed to combine the two feature types. Experimental results show that both our proposed appearance representation and its combination with geometry-based facial dynamics achieve favorable performances on four baseline databases： BBC, SPOS, MMI, and UvA-NEMO.
文摘三维局部约束模型(3D Constrained Local Model,CLM-Z)算法,综合利用灰度和深度信息检测三维人脸数据中的特征点(如眼角、鼻尖和嘴角),实现了较高的检测精度。CLM-Z方法一般使用人脸位置和平均三维人脸模型进行初始化。设计了四个实验定量地分析CLM-Z参数初始化对算法精度的影响:在BU-4DFE库上评估CLM-Z算法精度;通过平移人脸边界框扰动平移参数的初始值;通过缩放人脸边界框扰动尺度参数的初始值;通过给定绕y轴和z轴的旋转角扰动旋转参数的初始值。实验结果表明,CLM-Z算法可容忍平移扰动约为人脸宽的1/6,在(0.75,1.50)缩放范围内算法精度不会下降,可容忍y轴和z轴旋转角约20°。基于以上评估结果,进一步提出在纹理图像上检测特征点作为初始化,然后再进行CLM-Z迭代。在BU-4DFE数据库上的评估结果证明,该初始化方法能有效提升CLM-Z方法的特征点定位精度。
