Uncalibrated stereo visionđź”—
Similar to uncalibrated mosaicking, Halcon also provides an “uncalibrated” version of stereo vision, which derives information about the cameras from the scene itself by matching characteristic points. The main advantage of this method is that you need no special calibration object. The main disadvantage of this method is that, without a precisely known calibration object, the accuracy of the result is highly dependent on the content of the scene, i.e., the accuracy of the result degrades if the scene does not contain enough 3D information or if the extracted characteristic points in the two images do not precisely correspond to the same world points, e.g., due to occlusion.
In fact, Halcon provides two versions of uncalibrated stereo:
Uncalibrated stereo without knowledge about cameras and sceneđź”—
Without any knowledge about the cameras and about the scene, you can rectify the stereo images and perform a segmentation similar to the method described here.
For this, you first use the operator match_fundamental_matrix_ransac, which determines the so-called fundamental matrix.
This matrix models the cameras and their relation.
But in contrast to the model described for the stereo vision principle, internal and external parameters are not available separately.
Thus, no metric information can be derived.
The fundamental matrix is then passed on to the operator gen_binocular_proj_rectification, which is the “uncalibrated” version of the operator gen_binocular_rectification_map.
With the output of this operator, i.e., the rectification maps, you can then proceed to rectify the images.
Because the relative pose of the cameras is not known, you cannot generate the distance image and segment it as described here.
The example program %HALCONEXAMPLES%\hdevelop\Applications\Object-Recognition-2D\board_segmentation_uncalib.hdev shows an alternative approach that can be used if the reference plane appears dominant in the images, i.e., if many correspondences are found on it.
Because no calibration object is needed, this method can be used to perform stereo vision with a single camera. Note, however, that the method assumes that there are no lens distortions in the images. Therefore, the accuracy of the results degrades if the lens has significant distortions.
Uncalibrated stereo with known internal camera parametersđź”—
If the internal parameters of the camera are known, you can determine the relative pose between the cameras using the operator match_rel_pose_ransac and then use all the stereo methods described for the fully calibrated case.
There is, however, a limitation:
The determined pose is relative in a second sense, because it can be determined only up to a scale factor.
The reason for this is that without any knowledge about the scene, the algorithm cannot know whether the points in the scene are further away or whether the cameras are further apart because the effect in the image is the same in both cases.
If you have additional information about the scene, you can solve this ambiguity and determine the “real” relative pose.
This method is shown in the example program %HALCONEXAMPLES%\hdevelop\Applications\3D-Reconstruction\Binocular-Stereo\uncalib_stereo_boxes.hdev.