A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） ...A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.展开更多
This paper proposes a simple geometrical ray based approach to solve the stereo correspondence problem for the single-lens bi-prism stereovision system. Each image captured using this system can be divided into two su...This paper proposes a simple geometrical ray based approach to solve the stereo correspondence problem for the single-lens bi-prism stereovision system. Each image captured using this system can be divided into two sub-images on the left and right and these sub-images are generated by two virtual cameras which are produced by the bi-prism. This stereovision system is equivalent to the conventional two camera system and the two sub-images captured have disparities which can be used to reconstruct back the 3-dimensional （3D） scene. The stereo correspondence problem of this system will be solved geometrically by applying the epipolar geometry constraint on the generated virtual cameras instead of the real CCD camera. Experiments are conducted to validate the proposed method and the results are compared to the calibration based approach to confirm its accuracy and effectiveness.展开更多
Transformer-based stereo image super-resolution reconstruction(Stereo SR)methods have significantly improved image quality.However,existing methods have deficiencies in paying attention to detailed features and do not...Transformer-based stereo image super-resolution reconstruction(Stereo SR)methods have significantly improved image quality.However,existing methods have deficiencies in paying attention to detailed features and do not consider the offset of pixels along the epipolar lines in complementary views when integrating stereo information.To address these challenges,this paper introduces a novel epipolar line window attention stereo image super-resolution network(EWASSR).For detail feature restoration,we design a feature extractor based on Transformer and convolutional neural network(CNN),which consists of(shifted)window-based self-attention((S)W-MSA)and feature distillation and enhancement blocks(FDEB).This combination effectively solves the problem of global image perception and local feature attention and captures more discriminative high-frequency features of the image.Furthermore,to address the problem of offset of complementary pixels in stereo images,we propose an epipolar line window attention(EWA)mechanism,which divides windows along the epipolar direction to promote efficient matching of shifted pixels,even in pixel smooth areas.More accurate pixel matching can be achieved using adjacent pixels in the window as a reference.Extensive experiments demonstrate that our EWASSR can reconstruct more realistic detailed features.Comparative quantitative results show that in the experimental results of our EWASSR on the Middlebury and Flickr1024 data sets for 2×SR,compared with the recent network,the Peak signal-to-noise ratio(PSNR)increased by 0.37 dB and 0.34 dB,respectively.展开更多
The identification of the correspondences of points of views is an important task. A new feature matching algorithm for weakly calibrated stereo images of curved scenes is proposed, based on mere geometric constraints...The identification of the correspondences of points of views is an important task. A new feature matching algorithm for weakly calibrated stereo images of curved scenes is proposed, based on mere geometric constraints. After initial correspondences are built via the epipolar constraint, many point-to-point image mappings called homographies are set up to predict the matching position for feature points. To refine the predictions and reject false correspondences, four schemes are proposed. Extensive experiments on simulated data as well as on real images of scenes of variant depths show that the proposed method is effective and robust.展开更多
This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images ...This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.展开更多
This paper combines the least-square method and iteration method to get the fundamental matrix and develops a new evaluation function based on the epipolar geometry. During the iteration, with the evaluation function ...This paper combines the least-square method and iteration method to get the fundamental matrix and develops a new evaluation function based on the epipolar geometry. During the iteration, with the evaluation function as a measurment, the points which bring larger noise are deleted, and the points with smaller noise are retained, thus the precision of our method is increased. The experiment results indicate the new method is precise in calculation, stable in performance and resistant to noise.展开更多
Image-Based Rendering （IBR） is one powerful approach for generating virtual views. It can provide convincing animations without an explicit geometric representation. In this paper, several implementations of light f...Image-Based Rendering （IBR） is one powerful approach for generating virtual views. It can provide convincing animations without an explicit geometric representation. In this paper, several implementations of light field rendering are summa- rized from prior arts. Several characteristics, such as the regu- lar pattern in Epipolar Plane Images （EPIs）, of light field are explored with detail under 1D parallel camera arrangement. It is proved that it is quite efficient to synthesize virtual views for Super Multi-View （SMV） application, which is in the third phase of Free- Viewpoint Television （FTV）. In comparison with convolutional stereo matching method, in which the inter- mediate view is synthesized by the two adjacent views, light field rendering makes use of more views supplied to get the high-quality views.展开更多
Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed....Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.展开更多
基金supported by the Ministry of Education of Singapore under Grant No.R265-000-277-112
文摘A geometrical analysis based algorithm is proposed to achieve the stereo matching of a single-lens prism based stereovision system. By setting the multi- face prism in frontal position of the static CCD （CM-140MCL） camera, equivalent stereo images with different orientations are captured synchronously by virtual cameras which are defined by two boundary lines： the optical axis and CCD camera field of view boundary. Subsequently, the geometrical relationship between the 2D stereo images and corresponding 3D scene is established by employing two fundamentals： ray sketching in which all the pertinent points, lines, and planes are expressed in the 3D camera coordinates and the rule of refraction. Landing on this relationship, the epipolar geometry is thus obtained by fitting a set of corresponding candidate points and thereafter, stereo matching of the prism based stereovision system is obtained. Moreover, the unique geometrical properties of the imaging system allow the proposed method free from the complicated camera calibration procedures and to be easily generalized from binocular and tri-oeular to multi-ocular stereovision systems. The performance of the algorithm is presented through the experiments on the binocular imaging system and the comparison with a conventional projection method demonstrates the efficient assessment of our novel contributions.
文摘This paper proposes a simple geometrical ray based approach to solve the stereo correspondence problem for the single-lens bi-prism stereovision system. Each image captured using this system can be divided into two sub-images on the left and right and these sub-images are generated by two virtual cameras which are produced by the bi-prism. This stereovision system is equivalent to the conventional two camera system and the two sub-images captured have disparities which can be used to reconstruct back the 3-dimensional （3D） scene. The stereo correspondence problem of this system will be solved geometrically by applying the epipolar geometry constraint on the generated virtual cameras instead of the real CCD camera. Experiments are conducted to validate the proposed method and the results are compared to the calibration based approach to confirm its accuracy and effectiveness.
基金This work was supported by Sichuan Science and Technology Program(2023YFG0262).
文摘Transformer-based stereo image super-resolution reconstruction(Stereo SR)methods have significantly improved image quality.However,existing methods have deficiencies in paying attention to detailed features and do not consider the offset of pixels along the epipolar lines in complementary views when integrating stereo information.To address these challenges,this paper introduces a novel epipolar line window attention stereo image super-resolution network(EWASSR).For detail feature restoration,we design a feature extractor based on Transformer and convolutional neural network(CNN),which consists of(shifted)window-based self-attention((S)W-MSA)and feature distillation and enhancement blocks(FDEB).This combination effectively solves the problem of global image perception and local feature attention and captures more discriminative high-frequency features of the image.Furthermore,to address the problem of offset of complementary pixels in stereo images,we propose an epipolar line window attention(EWA)mechanism,which divides windows along the epipolar direction to promote efficient matching of shifted pixels,even in pixel smooth areas.More accurate pixel matching can be achieved using adjacent pixels in the window as a reference.Extensive experiments demonstrate that our EWASSR can reconstruct more realistic detailed features.Comparative quantitative results show that in the experimental results of our EWASSR on the Middlebury and Flickr1024 data sets for 2×SR,compared with the recent network,the Peak signal-to-noise ratio(PSNR)increased by 0.37 dB and 0.34 dB,respectively.
基金the Ph. D. Programs Foundation of Ministry of Education of China (20040248046).
文摘The identification of the correspondences of points of views is an important task. A new feature matching algorithm for weakly calibrated stereo images of curved scenes is proposed, based on mere geometric constraints. After initial correspondences are built via the epipolar constraint, many point-to-point image mappings called homographies are set up to predict the matching position for feature points. To refine the predictions and reject false correspondences, four schemes are proposed. Extensive experiments on simulated data as well as on real images of scenes of variant depths show that the proposed method is effective and robust.
基金funded by the Center for Unmanned Aircraft Systems(C-UAS)a National Science Foundation Industry/University Cooperative Research Center(I/UCRC)under NSF award Numbers IIP-1161036 and CNS-1650547along with significant contributions from C-UAS industry members。
文摘This paper introduces a new algorithm for estimating the relative pose of a moving camera using consecutive frames of a video sequence. State-of-the-art algorithms for calculating the relative pose between two images use matching features to estimate the essential matrix. The essential matrix is then decomposed into the relative rotation and normalized translation between frames. To be robust to noise and feature match outliers, these methods generate a large number of essential matrix hypotheses from randomly selected minimal subsets of feature pairs, and then score these hypotheses on all feature pairs. Alternatively, the algorithm introduced in this paper calculates relative pose hypotheses by directly optimizing the rotation and normalized translation between frames, rather than calculating the essential matrix and then performing the decomposition. The resulting algorithm improves computation time by an order of magnitude. If an inertial measurement unit(IMU) is available, it is used to seed the optimizer, and in addition, we reuse the best hypothesis at each iteration to seed the optimizer thereby reducing the number of relative pose hypotheses that must be generated and scored. These advantages greatly speed up performance and enable the algorithm to run in real-time on low cost embedded hardware. We show application of our algorithm to visual multi-target tracking(MTT) in the presence of parallax and demonstrate its real-time performance on a 640 × 480 video sequence captured on a UAV. Video results are available at https://youtu.be/Hh K-p2 h XNn U.
基金Supported by the National Science Foundation(69275004)the France-China Advanced Research Program
文摘This paper combines the least-square method and iteration method to get the fundamental matrix and develops a new evaluation function based on the epipolar geometry. During the iteration, with the evaluation function as a measurment, the points which bring larger noise are deleted, and the points with smaller noise are retained, thus the precision of our method is increased. The experiment results indicate the new method is precise in calculation, stable in performance and resistant to noise.
文摘Image-Based Rendering （IBR） is one powerful approach for generating virtual views. It can provide convincing animations without an explicit geometric representation. In this paper, several implementations of light field rendering are summa- rized from prior arts. Several characteristics, such as the regu- lar pattern in Epipolar Plane Images （EPIs）, of light field are explored with detail under 1D parallel camera arrangement. It is proved that it is quite efficient to synthesize virtual views for Super Multi-View （SMV） application, which is in the third phase of Free- Viewpoint Television （FTV）. In comparison with convolutional stereo matching method, in which the inter- mediate view is synthesized by the two adjacent views, light field rendering makes use of more views supplied to get the high-quality views.
基金The National Natural Science Founda-tion of China (No.60135020) and the National Defence Key Pre-research Project of China (No.413010701-3)
文摘Matching features such as curve segments in stereo images play a very important role in scene recomtruction. In this paper, a stereo matching algorithm for the trajectories composed of time stamped points is proposed. Based on time stamped points, planar curve match measurements are given first, such as time constraint, cross-ratio invariant constraint and eplpolar geometry constraint; then, a trajectory matching method is proposed based on epipolar geometry constraint and cross-ratio invariant constraint. In order to match the planar curve segments projected by perspective projection system, the curve start time and end time are selected first to prepare match candidates. Then, the epipolar equation is used to discard the unmatched curve segment candidates. At last, a cross ratio invariant constxaint is used to find the most matched curve segments. If their match measurement is higher than the specialized threshold, a candidate with the least cross ratio difference is then selected as the match result; otherwise, no match is found. Unlike the conventional planar curve segments matching algorithm, this paper presents a weakly calibrated binocular stereo vision system which is based on wide baseline. The stamped points are obtained by targets detecting method of flying objects from image sequences. Due to wide baseline, there must exist the projection not in epipolar monotonic order or the curve segments located in very short distance and keeping the epipolar monotonic order. By using the method mentioned above, experiments are made to match planar curve segments not only in epipolar monotonic order but also not in epipolar monotonic order. The results show that the performance of our curve matching algorithm is effective for matching the arc-like planar trajectories composed of time stamped points.
