摘要
针对现有深度学习光流计算方法的运动边缘模糊问题,提出了一种基于多尺度变形卷积的特征金字塔光流计算方法.首先,构造基于多尺度变形卷积的特征提取模型,显著提高图像边缘区域特征提取的准确性;然后,将多尺度变形卷积特征提取模型与特征金字塔光流计算网络耦合,提出一种基于多尺度变形卷积的特征金字塔光流计算模型;最后,设计一种结合图像与运动边缘约束的混合损失函数,通过指导模型学习更加精准的边缘信息,克服了光流计算运动边缘模糊问题.分别采用MPI-Sintel和KITTI2015测试图像集对该方法与代表性的深度学习光流计算方法进行综合对比分析.实验结果表明,该方法具有更高的光流计算精度,有效解决了光流计算的边缘模糊问题.
To cope with the issue of edge-blurring caused by the existing deep-learning based optical flow estimation methods,this paper proposes a feature pyramid optical flow estimation method based on multi-scale deformation convolution.Firstly,a feature extraction module based on multi-scale deformable convolution is constructed to improve the accuracy of feature extraction in the regions of image edges.Secondly,by coupling the multi-scale deformable convolution feature extraction module with the feature pyramid based optical flow estimation network,a feature pyramid optical flow estimation model based on multi-scale deformable convolution is presented.Thirdly,a hybrid loss function combining the constraints of image and motion edges is designed,which addresses the issue of edge-blurring by guiding the optical flow model to learn more accurate edge information.Finally,the MPI-Sintel and KITTI2015 test datasets are used for conducting a comprehensive comparison between the proposed method and some representative deep-learning based optical flow estimation methods.The experimental results indicate that the proposed method achieves higher computational accuracy,and overcomes the issue of edge-blurring in optical flow estimation.
作者
范兵兵
葛利跃
张聪炫
李兵
冯诚
陈震
FAN Bing-Bing;GE Li-Yue;ZHANG Cong-Xuan;LI Bing;FENG Cheng;CHEN Zhen(School of Measuring and Optical Engineering,Nanchang Hangkong University,Nanchang 330063;Key Laboratory of Jiangxi Province for Image Processing and Pattern Recognition,Nanchang Hangkong University,Nanchang 330063;National Key Laboratory of Pattern Recognition,Institute of Automation,Chinese Academy of Sciences,Beijing 100190;School of Instrumentation and Optoelectronic Engineering,Beihang University,Beijing 100083)
出处
《自动化学报》
EI
CAS
CSCD
北大核心
2023年第1期197-209,共13页
Acta Automatica Sinica
基金
科技创新2030“新一代人工智能”重大项目(2020AAA0105802,2020AAA0105801,2020AAA0105800)
国家重点研发计划(2020YFC2003800)
国家自然科学基金(62222206,62272209,61866026,61772255,61866025)
江西省技术创新引导类计划项目(20212AEI91005)
江西省教育厅科学技术项目(GJJ210910)
江西省优势科技创新团队(20165BCB19007)
江西省自然科学基金重点项目(20202ACB214007)
江西省图像处理与模式识别重点实验室开放基金(ET202104413)资助。
关键词
光流
深度学习
变形卷积
特征金字塔
边缘保护
Optical flow
deep-learning
deformable convolution
feature pyramid
edge-preserving
