Background Free-viewpoint video(FVV)is processed video content in which viewers can freely select the viewing position and angle.FVV delivers an improved visual experience and can also help synthesize special effects ...Background Free-viewpoint video(FVV)is processed video content in which viewers can freely select the viewing position and angle.FVV delivers an improved visual experience and can also help synthesize special effects and virtual reality content.In this paper,a complete FVV system is proposed to interactively control the viewpoints of video relay programs through multimedia terminals such as computers and tablets.Methods The hardware of the FVV generation system is a set of synchronously controlled cameras,and the software generates videos in novel viewpoints from the captured video using view interpolation.The interactive interface is designed to visualize the generated video in novel viewpoints and enable the viewpoint to be changed interactively.Results Experiments show that our system can synthesize plausible videos in intermediate viewpoints with a view range of up to 180°.展开更多
在自由视点视频系统中,如何能在视频终端得到高质量的视频图像已成为基于深度图的绘制(DIBR)技术所研究的主要任务,其中虚拟视点像素插值是该技术中影响绘制质量的一个重要环节。针对虚拟视点绘制标准方案中存在的问题,提出了一种基于...在自由视点视频系统中,如何能在视频终端得到高质量的视频图像已成为基于深度图的绘制(DIBR)技术所研究的主要任务,其中虚拟视点像素插值是该技术中影响绘制质量的一个重要环节。针对虚拟视点绘制标准方案中存在的问题,提出了一种基于空间加权的像素插值算法。它是通过对多个投影像素点的深度值和水平方向绝对距离进行加权操作来实现像素插值的。在插值过程中,该算法考虑了不同区域投影像素点个数对像素插值准确性的影响,从而剔除了部分失真像素点,并且在图像输出前还分别对左、右参考虚拟视点进行了失真检测和矫正。实验结果表明,该算法改善了绘制的主、客观质量,其中,PSNR平均提高0.30 d B,SSIM平均提高0.001 3。因此,该算法可以有效地抑制像素插值过程引入的噪声,提高像素插值的精度。展开更多
Image morphing is a powerful tool for visual effect. In this paper, a view interpolation algorithm is proposed to simulate a virtual walk along a street from start position to end position. To simulate a virtual walki...Image morphing is a powerful tool for visual effect. In this paper, a view interpolation algorithm is proposed to simulate a virtual walk along a street from start position to end position. To simulate a virtual walking view needs to create new appearing scene in the vision-vanishing point and disappearing scene beyond the scope of view. To attain these two aims we use two enhanced position parameters to match pixels of source images and target images. One enhanced position parameter is the angular coordinates of pixels. Another enhanced position parameter is the distances from pixels to the vision-vanishing point. According to the parameter values, pixels beyond the scope of view can be ＂moved＂ out in linear interpolation. Result demonstrates the validity of the algorithm. Another advantage of this algorithm is that the enhanced position parameters are based on real locations and walking distances, so it is also an approach to online virtual tour by satellite maps of virtual globe applications such as Google Earth.展开更多
In recent years, many image-based rendering techniques have advanced from static to dynamic scenes and thus become video-based rendering （VBR） methods. But actually, only a few of them can render new views on-line. ...In recent years, many image-based rendering techniques have advanced from static to dynamic scenes and thus become video-based rendering （VBR） methods. But actually, only a few of them can render new views on-line. We present a new VBR system that creates new views of a live dynamic scene. This system provides high quality images and does not require any background subtraction. Our method follows a plane-sweep approach and reaches real-time rendering using consumer graphic hardware, graphics processing unit （GPU）. Only one computer is used for both acquisition and rendering. The video stream acquisition is performed by at least 3 webcams. We propose an additional video stream management that extends the number of webcams to 10 or more. These considerations make our system low-cost and hence accessible for everyone. We also present an adaptation of our plane-sweep method to create simultaneously multiple views of the scene in real-time. Our system is especially designed for stereovision using autostereoscopic displays. The new views are computed from 4 webcams connected to a computer and are compressed in order to be transfered to a mobile phone. Using GPU programming, our method provides up to 16 images of the scene in real-time. The use of both GPU and CPU makes this method work on only one consumer grade computer.展开更多
目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)...目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355 d B,SSIM平均提高了0.001 15。结论改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。展开更多
Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 ...Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 D information,and 2 D semantic segmentation of the scene,to accomplish view synthesis of complicated scenes.We use the idea of cost volume to estimate the depth and confidence map of the scene,and use a multi-layer representation and resolution of the data to optimize the view synthesis of the main object.Results/Conclusions By applying different treatment methods on different layers of the volume,we can handle complicated scenes containing multiple persons and plentiful occlusions.We also propose the view-interpolation→multi-view reconstruction→view interpolation pipeline to iteratively optimize the result.We test our method on varying data of multi-view scenes and generate decent results.展开更多
基金Supported by USDA 2018-67021-27416NSFJS BK20190292.
文摘Background Free-viewpoint video(FVV)is processed video content in which viewers can freely select the viewing position and angle.FVV delivers an improved visual experience and can also help synthesize special effects and virtual reality content.In this paper,a complete FVV system is proposed to interactively control the viewpoints of video relay programs through multimedia terminals such as computers and tablets.Methods The hardware of the FVV generation system is a set of synchronously controlled cameras,and the software generates videos in novel viewpoints from the captured video using view interpolation.The interactive interface is designed to visualize the generated video in novel viewpoints and enable the viewpoint to be changed interactively.Results Experiments show that our system can synthesize plausible videos in intermediate viewpoints with a view range of up to 180°.
文摘在自由视点视频系统中,如何能在视频终端得到高质量的视频图像已成为基于深度图的绘制(DIBR)技术所研究的主要任务,其中虚拟视点像素插值是该技术中影响绘制质量的一个重要环节。针对虚拟视点绘制标准方案中存在的问题,提出了一种基于空间加权的像素插值算法。它是通过对多个投影像素点的深度值和水平方向绝对距离进行加权操作来实现像素插值的。在插值过程中,该算法考虑了不同区域投影像素点个数对像素插值准确性的影响,从而剔除了部分失真像素点,并且在图像输出前还分别对左、右参考虚拟视点进行了失真检测和矫正。实验结果表明,该算法改善了绘制的主、客观质量,其中,PSNR平均提高0.30 d B,SSIM平均提高0.001 3。因此,该算法可以有效地抑制像素插值过程引入的噪声,提高像素插值的精度。
文摘Image morphing is a powerful tool for visual effect. In this paper, a view interpolation algorithm is proposed to simulate a virtual walk along a street from start position to end position. To simulate a virtual walking view needs to create new appearing scene in the vision-vanishing point and disappearing scene beyond the scope of view. To attain these two aims we use two enhanced position parameters to match pixels of source images and target images. One enhanced position parameter is the angular coordinates of pixels. Another enhanced position parameter is the distances from pixels to the vision-vanishing point. According to the parameter values, pixels beyond the scope of view can be ＂moved＂ out in linear interpolation. Result demonstrates the validity of the algorithm. Another advantage of this algorithm is that the enhanced position parameters are based on real locations and walking distances, so it is also an approach to online virtual tour by satellite maps of virtual globe applications such as Google Earth.
基金This work was supported by Foundation of Technology Supporting the Creation of Digital Media Contents project (CREST, JST), Japan
文摘In recent years, many image-based rendering techniques have advanced from static to dynamic scenes and thus become video-based rendering （VBR） methods. But actually, only a few of them can render new views on-line. We present a new VBR system that creates new views of a live dynamic scene. This system provides high quality images and does not require any background subtraction. Our method follows a plane-sweep approach and reaches real-time rendering using consumer graphic hardware, graphics processing unit （GPU）. Only one computer is used for both acquisition and rendering. The video stream acquisition is performed by at least 3 webcams. We propose an additional video stream management that extends the number of webcams to 10 or more. These considerations make our system low-cost and hence accessible for everyone. We also present an adaptation of our plane-sweep method to create simultaneously multiple views of the scene in real-time. Our system is especially designed for stereovision using autostereoscopic displays. The new views are computed from 4 webcams connected to a computer and are compressed in order to be transfered to a mobile phone. Using GPU programming, our method provides up to 16 images of the scene in real-time. The use of both GPU and CPU makes this method work on only one consumer grade computer.
文摘目的基于深度图的绘制(DIBR)是一种新型的虚拟视点生成技术,在诸多方面得到了广泛的应用。然而,该技术还不能满足实时性的绘制需求。为了在保证绘制质量不下降的前提下,尽可能地提高绘制速度,提出了一种高效的3D-Warping(3维坐标变换)算法。方法主要在以下3个方面进行了改进:1)引入了深度—视差映射表技术,避免了重复地进行视差求取操作。2)对深度平坦的像素块进行基于块的3D-Warping,减少了映射的次数。对深度非平坦像素块中的像素点采取传统的基于像素点的3D-Warping,保证了映射的准确性。3)针对两种不同的3D-Warping方式,分别提出了相应的插值算法。在水平方向上,改进的像素插值算法对紧邻插值和Splatting(散射)插值算法进行了折中,只在映射像素点与待插值像素点很近的情况下才进行紧邻插值,否则进行Splatting插值;在深度方向上,它对Z-Buffer(深度缓存)技术进行了改进,舍弃了与前景物体太远的映射像素点,而对其他映射像素点按深度值进行加权操作。结果实验结果表明,与标准绘制方案的整像素精度相比,绘制时间平均节省了72.05%;与标准绘制方案的半像素精度相比,PSNR平均提高了0.355 d B,SSIM平均提高了0.001 15。结论改进算法非常适用于水平设置相机系统的DIBR技术中的整像素精度绘制,对包含大量深度平坦区域的视频序列效果明显,不但能够提高绘制的速度,而且可以有效地改善绘制的客观质量。
文摘Background Aiming at free-view exploration of complicated scenes,this paper presents a method for interpolating views among multi RGB cameras.Methods In this study,we combine the idea of cost volume,which represent 3 D information,and 2 D semantic segmentation of the scene,to accomplish view synthesis of complicated scenes.We use the idea of cost volume to estimate the depth and confidence map of the scene,and use a multi-layer representation and resolution of the data to optimize the view synthesis of the main object.Results/Conclusions By applying different treatment methods on different layers of the volume,we can handle complicated scenes containing multiple persons and plentiful occlusions.We also propose the view-interpolation→multi-view reconstruction→view interpolation pipeline to iteratively optimize the result.We test our method on varying data of multi-view scenes and generate decent results.
