摘要
快速实时生成表情逼真、姿态自然的虚拟人脸一直是较为有挑战性的研究。提出一种基于3DMM与GAN结合的实时人脸表情迁移方法。通过目标人脸的一段表演视频,将表演人员与目标人脸关键点建立映射关系,使用二维RGB摄像头实时跟踪表演人脸关键点并利用GAN生成目标虚拟人脸特征点,进一步估计人脸姿态。利用3DMM构成二维到三维人脸模型的重建,实时渲染出当前姿态的二维人脸表情,再将表演人脸表情与目标人脸表情进行融合,生成表情逼真的目标人脸。对比实验表明,该方法能得到更为逼真的人脸表情,可以模仿出目标人脸真实的表情,同时也能够达到实时性,在创建逼真的视频方面实现了更大的灵活性。同时,提出一种针对人脸表情迁移仿真效果的验证方法可以客观评价仿真人脸的结果。
It has been a challenging research to generate realistic expressions and natural gestures of virtual faces in realtime.This paper proposes a real-time facial expression transfer method based on 3DMM and GAN.It established a mapping relationship between the performer and the target face key point through a performance video of the target face.2D RGB camera was used to track the key points of performing face in real time,and the GAN was used to generate the virtual faces feature points of the target,so as to further estimate the face posture.And 3DMM was used to construct the reconstruction of 2D to 3D face models,and the 2D face expression of the current posture was rendered in real time.Then,the performing facial expression and the target facial expression were fused to generate a realistic target face.The comparative experiments show that this method can get a more realistic facial expression.It can imitate the real expression of the target face,and also achieve real-time and greater flexibility in creating realistic video.In addition,this paper proposes a verification method for the simulation effect of facial expression transfer,which can objectively evaluate the results of the simulation face.
作者
高翔
黄法秀
刘春平
陈虎
Gao Xiang;Huang Faxiu;Liu Chunping;Chen Hu(College of Computer(Software),Sichuan University,Chengdu 610065,Sichuan,China;Wisesoft Co.,Ltd,Chengdu 610045,Sichuan,China;National Key Laboratory of Fundamental Science on Synthetic Vision,Sichuan University,Chengdu 610064,Sichuan,China)
出处
《计算机应用与软件》
北大核心
2020年第4期119-126,共8页
Computer Applications and Software
基金
国家重点研发计划项目(2016YFC0801100)。
关键词
表情迁移
三维重建
表情融合
对抗生成
人脸检测
深度学习
Expression transfer
3D reconstruction
Expression fusion
Confrontation generation
Face detection
Deep learning
