摘要
目的图像修复是计算机视觉领域研究的一项重要内容,其目的是根据图像中已知内容来自动地恢复丢失的内容,在图像编辑、影视特技制作、虚拟现实及数字文化遗产保护等领域都具有广泛的应用价值。而近年来,随着深度学习在学术界和工业界的广泛研究,其在图像语义提取、特征表示、图像生成等方面的应用优势日益突出,使得基于深度学习的图像修复方法的研究成为了国内外一个研究热点,得到了越来越多的关注。为了使更多研究者对基于深度学习的图像修复理论及其发展进行探索,本文对该领域研究现状进行综述。方法首先对基于深度学习图像修复方法提出的理论依据进行分析;然后对其中涉及的关键技术进行研究;总结了近年来基于深度学习的主要图像修复方法,并依据修复网络的结构对现有方法进行了分类,即分为基于卷积自编码网络结构的图像修复方法、基于生成式对抗网络结构的图像修复方法和基于循环神经网络结构的图像修复方法。结果在基于深度学习的图像修复方法中,深度学习网络的设计和训练过程中的损失函数的选择是其重要的内容,各类方法各有优缺点和其适用范围,如何提高修复结果语义的合理性、结构及细节的正确性,一直是研究者们努力的方向,基于此目的,本文通过实验分析总结了各类方法的主要特点、存在的问题、对训练样本的要求、主要应用领域及参考代码。结论基于深度学习图像修复领域的研究已经取得了一些显著进展,但目前深度学习在图像修复中的应用仍处于起步阶段,主要研究的内容也仅仅是利用待修复图像本身的图像内容信息,因此基于深度学习的图像修复仍是一个极具挑战的课题。如何设计具有普适性的修复网络,提高修复结果的准确性,还需要更加深入的研究。
Objective Inpainting is the process of reconstructing lost or deteriorated parts of images and videos. This reconstruction process is an important research area in the field of computer vision,and its purpose is to automatically repair lost content according to the known content of the images and videos. Inpainting has extensive application value in the fields of image editing,film and television special effect production,virtual reality,and digital cultural heritage protection. Deep learning has been widely studied in the academic and industrial fields in recent years. Its advantages in image semantic extraction,feature representation,and image generation have become increasingly prominent,leading to the increasing attention to and popularity of research on image inpainting based on deep learning. This study reviews the current research status of image inpainting based on deep learning to enable researchers to explore its theory and development. Method This paper first discusses the issue of image inpainting and summarizes the advantages and disadvantages of the commonly used methods by comparing their application results in image restoration in large areas. The theoretical basis of image inpainting based on deep learning is then analyzed,and the key technologies of image inpainting based on deep learning,which include the generation network based on auto encoder,the general training methods of deep network,and the training methods based on convolutional auto encoder network,are studied. This paper also summarizes some image inpainting methods based on deep learning that have been proposed in recent years and classifies these methods into three categories according to the architecture of their repairing network: image inpainting methods based on deep convolutional auto encoder architecture,image inpainting methods based on generative adversarial network( GAN) architecture,and image inpainting based on recurrent neural network( RNN) architecture. The basic structure of the generation network based on autoencoder
作者
强振平
何丽波
陈旭
徐丹
Qiang Zhenping;He Libo;Chen Xu;Xu Dan(College of Big Data and Intelligent Engineering,Southwest Forestry University,Kunming 650224,China;School of Information Science and Engineering,Yunnan University,Kunming 650091,China)
出处
《中国图象图形学报》
CSCD
北大核心
2019年第3期447-463,共17页
Journal of Image and Graphics
基金
国家自然科学基金项目(11603016
61540062)
西南林业大学科研启动基金项目(111827)~~
关键词
图像修复
深度学习
卷积神经网络
生成式对抗网络
循环神经网络
深度卷积自编码器网络
image inpainting
deep learning
convolutional neural network
generative adversarial network
recurrent neural network
deep convolutional auto encoder network
