摘要
针对目前人脸缺损图像修复存在边缘模糊和失真等问题,提出了一种两阶段层次门控卷积网络(Hierarchical gated convolutional network,HGCN),并将其与边缘对抗网络相结合用于人脸图像修复。首先,采用边缘对抗网络生成边缘图像。其次,将边缘图、掩模和缺损图像作为输入,训练HGCN的GAN模型以修复缺损人脸图像。HGCN网络采用门控卷积取代传统卷积,并引入了扩张卷积,网络的主体由粗修复模块和精修复模块组成。在粗修复模块中,编码器和解码器网络结构用于粗修复;在精细修复模块中,引入注意力机制来增强特征提取能力,进一步细化修复结果。实验使用Celeba-HQ数据集和NVIDIA不规则掩码数据集作为训练数据集,采用门控卷积网络和注意力机制网络作为实验对比模型,PSNR、 SSIM和MAE作为实验评估指标。实验结果表明,对于缺损区域小于20%的人脸图像,所提出的网络在上述三个指标上优于两种比较网络,而对于缺失区域大于20%的图像,所提出的网络与两种比较方法性能指标接近。在视觉效果方面,所提出的方法在细节上也优于两种对比网络。因而,所提出的网络可以明显提高图像修复效果,尤其是对图像细节的修复效果。
Aiming at the problems of edge blur and distortion in the current damaged face image inpainting,a two-stage hierarchical gated convolutional network(HGCN) was proposed and then combined with edge adversarial network for face image inpainting.Firstly,the edge adversarial network was adopted to generate edge images.Secondly,the edge images,the masks and the occluded images were combined to train the generative adversarial network(GAN) model of the HGCN to generate the inpainted face images.In the HGCN,traditional convolution was replaced by gated convolution and the dilated convolution was introduced.The main structure of the HGCN is composed of coarse inpainting module and fine inpainting module.In the coarse inpainting module,the encoder and decoder network structure was used for coarse inpainting.In the fine inpainting module,the attention mechanism was introduced to enhance the feature extraction ability so as to further refine the inpainting results.In the experiment,the Celeba-HQ dataset and NVIDIA irregular mask dataset were used as the training datasets,the gated convolution network and attention module were adopted as comparing networks,and PSNR,SSIM and MAE were used as evaluation indicators.The experimental results demonstrated that for the face images with missing areas less than 20%,the proposed network works better than the two other networks on the above three indicators,and for the face images with missing areas greater than 20%,the proposed network is close to the comparison networks on three indicators.In terms of visual effects,the proposed method also surpasses the two contrasting networks in details.The proposed network can evidently improve the inpainting effect,especially image details.
作者
翟凤文
周钊
孙芳林
金静
ZHAI Fengwen;ZHOU Zhao;SUN Fanglin;JIN Jing(School of Electronics and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)
基金
supported by Natural Science Foundation of Gansu Province(No.21JR11RA062)
University Innovation Fund of Gansu Province(No.2022A-047)。
关键词
深度学习
人脸修复
层次门控卷积网络
边缘生成
生成对抗网络
deep learning
face inpainting
hierarchical gated convolutional network(HGCN)
edge confrontation
generative adversarial
