摘要
视频超分辨率技术在卫星遥感侦测、视频监控和医疗影像等方面发挥着关键作用,在各领域具有广阔的应用前景,受到广泛关注,但传统的视频超分辨率算法具有一定局限性。随着深度学习技术的愈发成熟,基于深度神经网络的超分辨率算法在性能上取得了长足进步。充分融合视频时空信息可以快速高效地恢复真实且自然的纹理,视频超分辨率算法因其独特的优势成为一个研究热点。本文系统地对基于深度学习的视频超分辨率的研究进展进行详细综述,对基于深度学习的视频超分辨率技术的数据集和评价指标进行全面归纳,将现有视频超分辨率方法按研究思路分成两大类,即基于图像配准的视频超分辨率方法和非图像配准的视频超分辨率方法,并进一步立足于深度卷积神经网络的模型结构、模型优化历程和运动估计补偿的方法将视频超分辨率网络细分为10个子类,同时利用充足的实验数据对每种方法的核心思想以及网络结构的优缺点进行了对比分析。尽管视频超分辨率网络的重建效果在不断优化,模型参数量在逐渐降低,训练和推理速度在不断加快,然而已有的网络模型在性能上仍然存在提升的潜能。本文对基于深度学习的视频超分辨率技术存在的挑战和未来的发展前景进行了讨论。
Video-related super-resolution(VSR)technique can be focused on high-resolution video profiling and restoration to optimize its low-resolution version-derived quality.It has been developing intensively in relevant to such domains like satellite remote sensing detection,video surveillance,medical imaging,and low-involved electronics.To reconstruct high-resolution frames,conventional video-relevant super-resolution methods can be used to estimate potential motion status and blur kernel parameters,which are challenged for multiscene hetegerneity.Due to the quick response ability of fully integrating video spatio-temporal information of real and natural textures,the emerging deep learning based video superresolution algorithms have been developing dramatically.We review and analyze current situation of deep learning based video super-resolution systematically and literately.First,popular YCbCr datasets are introduced like YUV25,YUV21,ultra video group(UVG),and the RGB datasets are involved in as well,such as video 4(Vid4),realistic and dynamic scenes(REDS),Vimeo90K.The profile information of each dataset is summarized,including its name,year of publication,number of videos,frame number,and resolution.Furthermore,key parameters of the video super-resolution algorithm are introduced in detail in terms of peak signal-to-noise ratio(PSNR),structural similarity(SSIM),video quality model for variable frame delay(VQM_VFD),and learned perceptual image patch similarity(LPIPS).For the concept of video super-resolution and single image super-resolution,the difference between video super-resolution and single image super-resolution can be shown and the former one has richer video frames-interrelated motion information.If the video is processed frame by frame in terms of the single image super-resolution method,there would be a large number of artifacts in the reconstructed video.We carry out deep learning based video super-resolution methods analysis and it has two key technical challenges of those are image alignment and feature
作者
江俊君
程豪
李震宇
刘贤明
王中元
Jiang Junjun;Cheng Hao;Li Zhenyu;Liu Xianming;Wang Zhongyuan(School of Computer Science and Technology,Harbin Institute of Technology,Harbin 150001,China;School of Computer,Wuhan University,Wuhan 430072,China)
出处
《中国图象图形学报》
CSCD
北大核心
2023年第7期1927-1964,共38页
Journal of Image and Graphics
基金
国家自然科学基金项目(61971165,92270116,62071339)。
