摘要
3维场景理解与重建技术能够使计算机对真实场景进行高精度复现并引导机器以3维空间的思维理解整个真实世界,从而使机器拥有足够智能参与到真实世界的生产与建设,并能通过场景的模拟为人类的决策和生活提供服务。3维场景理解与重建技术主要包含场景点云特征提取、扫描点云配准与融合、场景理解与语义分割、扫描物体点云补全与细粒度重建等,在处理真实扫描场景时,受到扫描设备、角度、距离以及场景复杂程度的影响,对技术的精准度和稳定性提出了更高的要求,相关的技术也十分具有挑战性。其中,原始扫描点云特征提取与配准融合旨在将同场景下多个扫描区域进行特征匹配,从而融合得到完整的场景点云,是理解与重建技术的基石;场景点云的理解与语义分割的目的在于对场景模型进行整体感知并根据语义特征划分为功能性物体甚至是部件的点云,是整套技术的核心组成部分;后续的物体点云细粒度补全主要研究扫描物体的结构恢复和残缺部分补全,是场景物体点云细粒度重建的关键性技术。本文围绕上述系列技术,详细分析了基于3维点云的场景理解与重建技术相关的应用领域和研究方向,归结总结了国内外的前沿进展与研究成果,对未来的研究方向和技术发展进行了展望。
3D scene understanding and reconstruction are essential for machine vision and intelligence,which aim to reconstruct completed models of real scenes from multiple scene scans and understand the semantic meanings of each functional component in the scene.This technique is indispensable for real world digitalization and simulation,which can be widely used in related domains like robots,navigation system and virtual tourism.Its key challenges are required to be resolved on the three aspects:1) to recognize the same area in multiple real scans and fuse all the scans into an integrated scene point cloud;2) to make sense of the whole scene and recognize the semantics of multiple functional components;3) to complete the missing region in the original point cloud caused by occlusion during scanning.It is necessary to extract point cloud feature in order to fuse multiple real scene scans into an integrated point cloud,which can be invariant to scanning position and rotation.Thus,intrinsic geometry features like point distance and singular value in neighborhood covariance matrix are often involved in rotation-invariant feature design.Contrastive learning scheme is usually taken to help the learned features from the same area to be close to each other,while extracted features from different areas to be far away.To get generalization ability better,data augmentation of scanned point cloud can also be used during feature learning process.Features-learnt pose estimation of scanning device can be configured to calculate the transformation matrix between point cloud pairs.After the transformation relationship is sorted out,the following point cloud fusion can be implemented using the raw point cloud scans.To further understand raw point cloud-based whole scene and segment the whole scene into functional parts on the basis of multiple semantics,an effective and efficient network with appropriate 3D convolution operation is required to parse entire points-based scene hierarchically,and specific learning schemes are necessary as we
作者
龚靖渝
楼雨京
柳奉奇
张志伟
陈豪明
张志忠
谭鑫
谢源
马利庄
Gong Jingyu;Lou Yujing;Liu Fengqi;Zhang Zhiwei;Chen Haoming;Zhang Zhizhong;Tan Xin;Xie Yuan;Ma Lizhuang(Department of Computer Science and Engineering,Shanghai Jiao Tong University,Shanghai 200240,China;School of Computer Science and Technology,East China Normal University,Shanghai 200062,China)
出处
《中国图象图形学报》
CSCD
北大核心
2023年第6期1741-1766,共26页
Journal of Image and Graphics
基金
国家自然科学基金项目(61972157,72192821)
上海市科技创新行动计划人工智能科技支撑项目(21511101200)。
关键词
3维场景
点云融合
场景分割
物体形状补全
深度学习
3D scenes
point could fusion
scene segmentation
object shape completion
deep learning
