摘要
目的近年来,深度卷积神经网络成为单帧图像超分辨率重建任务中的研究热点。针对多数网络结构均是采用链式堆叠方式使得网络层间联系弱以及分层特征不能充分利用等问题,提出了多阶段融合网络的图像超分辨重建方法,进一步提高重建质量。方法首先利用特征提取网络得到图像的低频特征,并将其作为两个子网络的输入,其一通过编码网络得到低分辨率图像的结构特征信息,其二通过阶段特征融合单元组成的多路径前馈网络得到高频特征,其中融合单元将网络连续几层的特征进行融合处理并以自适应的方式获得有效特征。然后利用多路径连接的方式连接不同的特征融合单元以增强融合单元之间的联系,提取更多的有效特征,同时提高分层特征的利用率。最后将两个子网络得到的特征进行融合后,利用残差学习完成高分辨图像的重建。结果在4个基准测试集Set5、Set14、B100和Urban100上进行实验,其中放大规模为4时,峰值信噪比分别为31. 69 d B、28. 24 d B、27. 39 d B和25. 46 d B,相比其他方法的结果具有一定提升。结论本文提出的网络克服了链式结构的弊端,通过充分利用分层特征提取更多的高频信息,同时利用低分辨率图像本身携带的结构特征信息共同完成重建,并取得了较好的重建效果。
Objective Image super-resolution is an important branch of digital image processing and computer vision.This method has been widely used in video surveillance,medical imaging,and security and surveillance imaging in recent years.Super-resolution aims to reconstruct a high-resolution image from an observed degraded low-resolution one.Early methods include interpolation,neighborhood embedding,and sparse coding.Deep convolutional neural network has recently become a major research topic in the field of single image super-resolution reconstruction.This network can learn the mapping between high-and low-resolution images better than traditional learning-based methods.However,many deep learningbased methods present two evident drawbacks.First,most methods use chained stacking to create the network.Each layer of the network is only related to its previous layer,leading to weak inter-layer relationships.Second,the hierarchical features of the network are partially utilized.These shortcomings can lead to loss of high frequency components.A novel image super-resolution reconstruction method based on multi-staged fusion network is proposed to address these drawbacks.This method is used to improve the quality of image reconstruction.Method Numerous studies have shown that feature re-usage can improve the capability of the network to extract and express features.Thus,our research is based on the idea of feature re-usage.We implemented this idea through the multipath connection,which includes two forms,namely,global multipath mode and local fusion unit.First,the proposed model uses an interpolated low-resolution image as input.The feature extraction network extracts shallow features as the mixture network’s input.Mixture network consists of two parts.The first one is pixel encoding network,which is used to obtain structural feature information of the image.This network presents four weight layers,each consisting of 64 filters with a size of 1×1,which can guarantee that the feature map distribution will be protected.This pro
作者
沈明玉
俞鹏飞
汪荣贵
杨娟
薛丽霞
Shen Mingyu;Yu Pengfei;Wang Ronggui;Yang Juan;Xue Lixia(School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230601,China)
出处
《中国图象图形学报》
CSCD
北大核心
2019年第8期1258-1269,共12页
Journal of Image and Graphics
关键词
卷积神经网络
超分辨率重建
分层特征
阶段特征融合
多路径连接
convolutional neural network(CNN)
super-resolution reconstructions
hierarchical features
staged feature fusion
multi-path mode
