摘要
目的雨天户外采集的图像常常因为雨线覆盖图像信息产生色变和模糊现象。为了提高雨天图像的质量,本文提出一种基于自适应选择卷积网络深度学习的单幅图像去雨算法。方法针对雨图中背景误判和雨痕残留问题,加入网络训练的雨线修正系数(refine factor,RF),改进现有雨图模型,更精确地描述雨图中各像素受到雨线的影响。构建选择卷积网络(selective kernel network,SK Net),自适应地选择不同卷积核对应维度的信息,进一步学习、融合不同卷积核的信息,提高网络的表达力,最后构建包含SK Net、refine factor net和residual net子网络的自适应卷积残差修正网络(selective kernel convolution using a residual refine factor,SKRF),直接学习雨线图和残差修正系数(RF),减少映射区间,减少背景误判。结果实验通过设计的SKRF网络,在公开的Rain12测试集上进行去雨实验,取得了比现有方法更高的精确度,峰值信噪比(peak signal to noise ratio,PSNR)达到34.62 d B,结构相似性(structural similarity,SSIM)达到0.9706。表明SKRF网络对单幅图像去雨效果有明显优势。结论单幅图像去雨SKRF算法为雨图模型中的雨线图提供一个额外的修正残差系数,以降低学习映射区间,自适应选择卷积网络模型提升雨图模型的表达力和兼容性。
Objective Rain lines adversely affect the visual quality of images collected from outdoors. Severe weather conditions,such as rain,fog,and haze,can affect the quality of these images and make them unusable. These degraded images may also drastically affect the performance of man’s vision system. Given that rain is a common meteorological phenomenon,an algorithm that can remove rain from single image is of practical significance. Given that video-based de-raining methods obtain pixel information of the same location at different periods,removing rain from an individual image is more challenging because of less available information. Traditional de-raining methods mainly focus on rain map modeling and use mathematical optimization to detect and remove rain streaks,but the performance of such approach requires further improvement. Method To address the above problems,this paper establishes a convolution neural network for single image rain removal that is trained on a synthetic dataset. The contributions of this work are as follows. 1) To expand the neural receptive field of a convolution neural network that learns abstract feature representation of rain streaks and the ground truth,this work establishes a selective kernel network based on multi-scale convolution with different kernel for feature learning. To accomplish useful information fusion and selection,an external non-linear weight learning mechanism is developed to redistribute the weight for the corresponding channel’s feature information from different convolution kernels. This mechanism enables the network to select the feature information of different receptive fields adaptively and enhance its expression ability and rain removal capability. 2) The existing rain map model shows some limitations at the training stage.Completing this model by adding a learnable refine factor that modifies each pixel in a rain streak image,can enhance the accuracy of the result and prevent background misjudgment. The range of the refining factor is also limited to reduc
作者
王美华
何海君
李超
Wang Meihua;He Haijun;Li Chao(College of Mathematics and Informatics,South China Agricultural University,Guangzhou 510642,China)
出处
《中国图象图形学报》
CSCD
北大核心
2020年第12期2484-2493,共10页
Journal of Image and Graphics
基金
国家自然科学基金项目(61976052)
广东省基础与应用基础研究基金项目(2019B1515210009)。
关键词
单幅图像去雨
深度学习
选择卷积网络
修正系数
残差学习
single image rain removal
deep learning
selective kernel network(SK Net)
refine factor(RF)
residual learning
