A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing m...A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.展开更多
Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes ...Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes up 90% time-consuming in typical motion estimation-based depth-map generation algorithms. In order to reduce the computational complexity, in this paper a new fast depth-map generation algorithm based on motion search is developed, in which a fast diamond search algorithm is adopted to decide whether a 16x16 or 4x4 block size is used based on Sobel operator in the motion search module to obtain a sub-depth-map. Then the sub-depth-map will be fused with the sub-depth-maps gotten from depth from color component Cr and depth from linear perspective modules to compensate and refine detail of the depth-map, finally obtain a better depth-map. The simulation results demonstrate that the new approach can greatly reduce over 50% computational complexity compared to other existing methods.展开更多
Recently, the popularity of 3D content is on the rise because of its immersive experience to view- ers. While demands for 3D contents and 3D technologies increase, only a few copyright protection methods for 3D conten...Recently, the popularity of 3D content is on the rise because of its immersive experience to view- ers. While demands for 3D contents and 3D technologies increase, only a few copyright protection methods for 3D contents have been proposed. The simplest infringement is the illegal distribution of the single 2D image from 3D content. The leaked image is still valuable as 2D content and the leakage can be occurred in DIBR system. To detect the leaked image, we focus on the hole-filled region which is caused by the hole-filling procedure mandatory in DIBR system. To estimate the hole-filled regions, two different procedures are conducted to extract edges and to estimate 3D warping traces, respectively. After that, the hole-filled regions are estimated and the left-right-eye image discrimination (LR discrimination) is also conducted. Experimental results demonstrate the effectiveness of the proposed method using quantitative measures.展开更多
DIBR(Depth Image Based Rendering)算法的提出,使得3D视频只需要一个原始视点的纹理信息及其对应的深度信息就能绘制出新的虚拟视点,因此深度信息的精准与否将直接影响到绘制的虚拟视点质量。由于深度失真经常发生在前景与背景的过渡区...DIBR(Depth Image Based Rendering)算法的提出,使得3D视频只需要一个原始视点的纹理信息及其对应的深度信息就能绘制出新的虚拟视点,因此深度信息的精准与否将直接影响到绘制的虚拟视点质量。由于深度失真经常发生在前景与背景的过渡区域,本文对该区域像素定义为过渡像素,针对这种过渡像素的深度失真问题进行了分析,并提出了一种基于前景的过渡像素深度划归方法。主要思想是:通过阈值判断找出过渡像素的位置,然后将前景的深度值赋予过渡像素。实验表明,这种方法能够有效的减少绘制过程的失真,提高虚拟视点的质量。展开更多
In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple te...In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple textures and depth maps in multi-view camera configuration. This method solves the cracks and holes problem due to sampling rate by performing an inverse warping to retrieve texture images. This step allows a simple and accurate re-sampling of synthetic pixels. To enforce the spatial consistency of color and remove the pixels wrapped incorrectly because of inaccuracy depth maps, we propose some processing steps. The warped depth and warped texture images are used to classify pixels as stable, unstable and disoccluded pixels. The stable pixels are used to create an initial new view by weighted interpolation. To refine the new view, Graph cuts are used to select the best candidates for each unstable pixel. Finally, the remaining disoccluded regions are filled by our inpainting method based on depth information and texture neighboring pixel values. Our experiment on several multi-view data sets is encouraging in both subjective and objective results. Furthermore, our proposal can flexibly use more than two views in multi-view system to create a new view with higher quality.展开更多
In free viewpoint video(FVV)and 3DTV,the depth image-based rendering method has been put forward for rendering virtual view video based on multi-view video plus depth(MVD) format.However,the projection with slightly d...In free viewpoint video(FVV)and 3DTV,the depth image-based rendering method has been put forward for rendering virtual view video based on multi-view video plus depth(MVD) format.However,the projection with slightly different perspective turns the covered background regions into hole regions in the rendered video.This paper presents a depth enhanced image summarization generation model for the hole-filling via exploiting the texture fidelity and the geometry consistency between the hole and the remaining nearby regions.The texture fidelity and the geometry consistency are enhanced by drawing texture details and pixel-wise depth information into the energy cost of similarity measure correspondingly.The proposed approach offers significant improvement in terms of 0.2dB PSNR gain,0.06 SSIM gain and subjective quality enhancement for the hole-filling images in virtual viewpoint video.展开更多
基金Project supported by the National Natural Science Foundation of China (No 60802013)the Natural Science Foundation of Zhe-jiang Province, China (No Y106574)
文摘A new algorithm is proposed for restoring disocclusion regions in depth-image-based rendering （DIBR） warped images. Current solutions include layered depth image （LDI）, pre-filtering methods, and post-processing methods. The LDI is complicated, and pre-filtering of depth images causes noticeable geometrical distortions in cases of large baseline warping. This paper presents a depth-aided inpainting method which inherits merits from Criminisi＇s inpainting algorithm. The proposed method features incorporation of a depth cue into texture estimation. The algorithm efficiently handles depth ambiguity by penalizing larger Lagrange multipliers of flling points closer to the warping position compared with the surrounding existing points. We perform morphological operations on depth images to accelerate the algorithm convergence, and adopt a luma-first strategy to adapt to various color sampling formats. Experiments on test multi-view sequence showed that our method has superiority in depth differentiation and geometrical loyalty in the restoration of warped images. Also, peak signal-to-noise ratio （PSNR） statistics on non-hole regions and whole image comparisons both compare favorably to those obtained by state of the art techniques.
文摘Generation of a depth-map from 2D video is the kernel of DIBR (Depth Image Based Rendering) in 2D-3D video conversion systems. However it occupies over most of the system resource where the motion search module takes up 90% time-consuming in typical motion estimation-based depth-map generation algorithms. In order to reduce the computational complexity, in this paper a new fast depth-map generation algorithm based on motion search is developed, in which a fast diamond search algorithm is adopted to decide whether a 16x16 or 4x4 block size is used based on Sobel operator in the motion search module to obtain a sub-depth-map. Then the sub-depth-map will be fused with the sub-depth-maps gotten from depth from color component Cr and depth from linear perspective modules to compensate and refine detail of the depth-map, finally obtain a better depth-map. The simulation results demonstrate that the new approach can greatly reduce over 50% computational complexity compared to other existing methods.
文摘Recently, the popularity of 3D content is on the rise because of its immersive experience to view- ers. While demands for 3D contents and 3D technologies increase, only a few copyright protection methods for 3D contents have been proposed. The simplest infringement is the illegal distribution of the single 2D image from 3D content. The leaked image is still valuable as 2D content and the leakage can be occurred in DIBR system. To detect the leaked image, we focus on the hole-filled region which is caused by the hole-filling procedure mandatory in DIBR system. To estimate the hole-filled regions, two different procedures are conducted to extract edges and to estimate 3D warping traces, respectively. After that, the hole-filled regions are estimated and the left-right-eye image discrimination (LR discrimination) is also conducted. Experimental results demonstrate the effectiveness of the proposed method using quantitative measures.
文摘DIBR(Depth Image Based Rendering)算法的提出,使得3D视频只需要一个原始视点的纹理信息及其对应的深度信息就能绘制出新的虚拟视点,因此深度信息的精准与否将直接影响到绘制的虚拟视点质量。由于深度失真经常发生在前景与背景的过渡区域,本文对该区域像素定义为过渡像素,针对这种过渡像素的深度失真问题进行了分析,并提出了一种基于前景的过渡像素深度划归方法。主要思想是:通过阈值判断找出过渡像素的位置,然后将前景的深度值赋予过渡像素。实验表明,这种方法能够有效的减少绘制过程的失真,提高虚拟视点的质量。
文摘In three-dimensional television (3DTV), an interactive free viewpoint selection application has received more attention so far. This paper presents a novel method that synthesizes a free-viewpoint based on multiple textures and depth maps in multi-view camera configuration. This method solves the cracks and holes problem due to sampling rate by performing an inverse warping to retrieve texture images. This step allows a simple and accurate re-sampling of synthetic pixels. To enforce the spatial consistency of color and remove the pixels wrapped incorrectly because of inaccuracy depth maps, we propose some processing steps. The warped depth and warped texture images are used to classify pixels as stable, unstable and disoccluded pixels. The stable pixels are used to create an initial new view by weighted interpolation. To refine the new view, Graph cuts are used to select the best candidates for each unstable pixel. Finally, the remaining disoccluded regions are filled by our inpainting method based on depth information and texture neighboring pixel values. Our experiment on several multi-view data sets is encouraging in both subjective and objective results. Furthermore, our proposal can flexibly use more than two views in multi-view system to create a new view with higher quality.
文摘In free viewpoint video(FVV)and 3DTV,the depth image-based rendering method has been put forward for rendering virtual view video based on multi-view video plus depth(MVD) format.However,the projection with slightly different perspective turns the covered background regions into hole regions in the rendered video.This paper presents a depth enhanced image summarization generation model for the hole-filling via exploiting the texture fidelity and the geometry consistency between the hole and the remaining nearby regions.The texture fidelity and the geometry consistency are enhanced by drawing texture details and pixel-wise depth information into the energy cost of similarity measure correspondingly.The proposed approach offers significant improvement in terms of 0.2dB PSNR gain,0.06 SSIM gain and subjective quality enhancement for the hole-filling images in virtual viewpoint video.
