Operator Reference

binocular_distanceT_binocular_distanceBinocularDistanceBinocularDistancebinocular_distance (Operator)

binocular_distanceT_binocular_distanceBinocularDistanceBinocularDistancebinocular_distance — Compute the distance values for a rectified stereo image pair using correlation techniques.

Signature

binocular_distance(ImageRect1, ImageRect2 : Distance, Score : CamParamRect1, CamParamRect2, RelPoseRect, Method, MaskWidth, MaskHeight, TextureThresh, MinDisparity, MaxDisparity, NumLevels, ScoreThresh, Filter, SubDistance : )

Description

binocular_distancebinocular_distanceBinocularDistanceBinocularDistancebinocular_distance computes the distance values for a rectified stereo image pair using correlation techniques. The operator first calculates the disparities between the two images ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 similar to binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity. The resulting disparities are transformed into distance values of the corresponding 3D world points to the rectified stereo camera system as in disparity_to_distancedisparity_to_distanceDisparityToDistanceDisparityToDistancedisparity_to_distance. The distances are returned in the single-channel image DistanceDistanceDistancedistancedistance in which each gray value represents the distance of the respective 3D world point to the stereo camera system.

The algorithm requires a reference image ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and a search image ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 which must be rectified, i.e., corresponding epipolar lines are parallel and lie on identical image rows (). In case this assumption is violated the images can be rectified by using the operators calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerascalibrate_cameras, gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map and map_imagemap_imageMapImageMapImagemap_image. Hence, given a pixel in the reference image ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 the homologous pixel in ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 is selected by searching along the corresponding row in ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 and matching a local neighborhood within a rectangular window of size MaskWidthMaskWidthMaskWidthmaskWidthmask_width and MaskHeightMaskHeightMaskHeightmaskHeightmask_height. For each defined reference pixel the pixel correspondences are transformed into distances of the world points defined by the intersection of the lines of sight of a corresponding pixel pair to the plane of the rectified stereo system.

For this transformation the rectified internal camera parameters CamParamRect1CamParamRect1CamParamRect1camParamRect1cam_param_rect_1 of camera 1 and CamParamRect2CamParamRect2CamParamRect2camParamRect2cam_param_rect_2 of camera 2, and the pose with the external parameters RelPoseRectRelPoseRectRelPoseRectrelPoseRectrel_pose_rect have to be defined. Latter one is of the form and characterizes the relative pose of both cameras to each other. More precisely, it specifies the point transformation from the rectified camera system 2 (ccsR2) into the rectified camera system 1 (ccsR1), see Transformations / Poses and “Solution Guide III-C - 3D Vision”. These parameters can be obtained from the operator calibrate_camerascalibrate_camerasCalibrateCamerasCalibrateCamerascalibrate_cameras and gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map. After all, a quality measure for each distance value is returned in ScoreScoreScorescorescore, containing the best result of the matching function of a reference pixel. For the matching, the gray values of the original unprocessed images are used.

'sad'"sad""sad""sad""sad": Summed Absolute Differences with .
'ssd'"ssd""ssd""ssd""ssd": Summed Squared Differences with .
'ncc'"ncc""ncc""ncc""ncc": Normalized Cross Correlation with .

with

, , , : row and column coordinates of the corresponding pixels of the two input images,

, : gray values of the unprocessed input images,

: size of correlation window

: mean value within the correlation window of width and height .

Note that the methods 'sad'"sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd" compare the gray values of the pixels within a mask window directly, whereas 'ncc'"ncc""ncc""ncc""ncc" compensates for the mean gray value and its variance within the mask window. Therefore, if the two images differ in brightness and contrast, this method should be preferred. For images with similar brightness and contrast 'sad'"sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd" are to be preferred as they are faster because of less complex internal computations. See binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity for further details.

It should be noted that the quality of correlation for rising is falling in methods 'sad'"sad""sad""sad""sad" and 'ssd'"ssd""ssd""ssd""ssd" (the best quality value is 0) but rising in method 'ncc'"ncc""ncc""ncc""ncc" (the best quality value is 1.0).

The size of the correlation window ( and ) has to be odd numbered and is passed in MaskWidthMaskWidthMaskWidthmaskWidthmask_width and MaskHeightMaskHeightMaskHeightmaskHeightmask_height. The search space is confined by the minimum and maximum disparity value MinDisparityMinDisparityMinDisparityminDisparitymin_disparity and MaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity. Due to pixel values not defined beyond the image border the resulting domain of DistanceDistanceDistancedistancedistance and ScoreScoreScorescorescore is generally not set along the image border within a margin of height MaskHeightMaskHeightMaskHeightmaskHeightmask_height/2 at the top and bottom border and of width MaskWidthMaskWidthMaskWidthmaskWidthmask_width/2 at the left and right border. For the same reason, the maximum disparity range is reduced at the left and right image border.

Since matching turns out to be highly unreliable when dealing with poorly textured areas, the minimum variance within the correlation window can be defined in TextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh. This threshold is applied on both input images ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 and ImageRect2ImageRect2ImageRect2imageRect2image_rect_2. In addition, ScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh guarantees the matching quality and defines the maximum ('sad'"sad""sad""sad""sad",'ssd'"ssd""ssd""ssd""ssd") or, respectively, minimum ('ncc'"ncc""ncc""ncc""ncc") score value of the correlation function. Setting FilterFilterFilterfilterfilter to 'left_right_check'"left_right_check""left_right_check""left_right_check""left_right_check", moreover, increases the robustness of the returned matches, as the result relies on a concurrent direct and reverse match, whereas 'none'"none""none""none""none" switches it off.

The number of pyramid levels used to improve the time response of binocular_distancebinocular_distanceBinocularDistanceBinocularDistancebinocular_distance is determined by NumLevelsNumLevelsNumLevelsnumLevelsnum_levels. Following a coarse-to-fine scheme disparity images of higher levels are computed and segmented into rectangular subimages to reduce the disparity range on the next lower pyramid level. TextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh and ScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh are applied on every level and the returned domain of the DistanceDistanceDistancedistancedistance and ScoreScoreScorescorescore images arises from the intersection of the resulting domains of every single level. Generally, pyramid structures are the more advantageous the more the distance image can be segmented into regions of homogeneous distance values and the bigger the disparity range must be specified. As a drawback, coarse pyramid levels might loose important texture information which can result in deficient distance values.

Finally, the value 'interpolation'"interpolation""interpolation""interpolation""interpolation" for parameter SubDistanceSubDistanceSubDistancesubDistancesub_distance increases the refinement and accuracy of the distance values. It is switched off by setting the parameter to 'none'"none""none""none""none".

Attention

If using cameras with telecentric lenses, the DistanceDistanceDistancedistancedistance is not defined as the distance of a point to the camera but as the distance from the point to the plane, defined by the y-axes of both cameras and their baseline (see gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map).

For a stereo setup of mixed type (i.e., for a stereo setup in which one of the original cameras is a perspective camera and the other camera is a telecentric camera; see gen_binocular_rectification_mapgen_binocular_rectification_mapGenBinocularRectificationMapGenBinocularRectificationMapgen_binocular_rectification_map), the rectifying plane of the two cameras is in a position with respect to the object that would lead to very unintuitive distances. Therefore, binocular_distancebinocular_distanceBinocularDistanceBinocularDistancebinocular_distance does not support a stereo setup of mixed type. For stereo setups of mixed type, please use reconstruct_surface_stereoreconstruct_surface_stereoReconstructSurfaceStereoReconstructSurfaceStereoreconstruct_surface_stereo, in which the reference coordinate system can be chosen arbitrarily. Alternatively, binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity and disparity_image_to_xyzdisparity_image_to_xyzDisparityImageToXyzDisparityImageToXyzdisparity_image_to_xyz might be used.

Additionally, stereo setups that contain cameras with and without hypercentric lenses at the same time are not supported.

Execution Information

Multithreading type: reentrant (runs in parallel with non-exclusive operators).
Multithreading scope: global (may be called from any thread).
Automatically parallelized on internal data level.

Parameters

ImageRect1ImageRect1ImageRect1imageRect1image_rect_1 (input_object) singlechannelimage → object (byte)

Rectified image of camera 1.

ImageRect2ImageRect2ImageRect2imageRect2image_rect_2 (input_object) singlechannelimage → object (byte)

Rectified image of camera 2.

DistanceDistanceDistancedistancedistance (output_object) singlechannelimage → object (real)

Distance image.

ScoreScoreScorescorescore (output_object) singlechannelimage → object (real)

Evaluation of a distance value.

CamParamRect1CamParamRect1CamParamRect1camParamRect1cam_param_rect_1 (input_control) campar → (real / integer / string)

Internal camera parameters of the rectified camera 1.

CamParamRect2CamParamRect2CamParamRect2camParamRect2cam_param_rect_2 (input_control) campar → (real / integer / string)

Internal camera parameters of the rectified camera 2.

RelPoseRectRelPoseRectRelPoseRectrelPoseRectrel_pose_rect (input_control) pose → (real / integer)

Point transformation from the rectified camera 2 to the rectified camera 1.

Number of elements: 7

MethodMethodMethodmethodmethod (input_control) string → (string)

Matching function.

Default: 'ncc' "ncc" "ncc" "ncc" "ncc"

List of values: 'ncc'"ncc""ncc""ncc""ncc", 'sad'"sad""sad""sad""sad", 'ssd'"ssd""ssd""ssd""ssd"

MaskWidthMaskWidthMaskWidthmaskWidthmask_width (input_control) integer → (integer)

Width of the correlation window.

Default: 11

Suggested values: 5, 7, 9, 11, 21

Restriction: 3 <= MaskWidth && odd(MaskWidth)

MaskHeightMaskHeightMaskHeightmaskHeightmask_height (input_control) integer → (integer)

Height of the correlation window.

Default: 11

Suggested values: 5, 7, 9, 11, 21

Restriction: 3 <= MaskHeight && odd(MaskHeight)

TextureThreshTextureThreshTextureThreshtextureThreshtexture_thresh (input_control) real → (real / integer)

Variance threshold of textured image regions.

Default: 0.0

Suggested values: 0.0, 2.0, 5.0, 10.0

Restriction: 0.0 <= TextureThresh

MinDisparityMinDisparityMinDisparityminDisparitymin_disparity (input_control) integer → (integer)

Minimum of the expected disparities.

Default: 0

Value range: -32768 ≤ MinDisparity MinDisparity MinDisparity minDisparity min_disparity ≤ 32767

MaxDisparityMaxDisparityMaxDisparitymaxDisparitymax_disparity (input_control) integer → (integer)

Maximum of the expected disparities.

Default: 30

Value range: -32768 ≤ MaxDisparity MaxDisparity MaxDisparity maxDisparity max_disparity ≤ 32767

NumLevelsNumLevelsNumLevelsnumLevelsnum_levels (input_control) integer → (integer)

Number of pyramid levels.

Default: 1

Suggested values: 1, 2, 3, 4

Restriction: 1 <= NumLevels

ScoreThreshScoreThreshScoreThreshscoreThreshscore_thresh (input_control) real → (real / integer)

Threshold of the correlation function.

Default: 0.0

Suggested values: 0.0, 2.0, 5.0, 10.0

FilterFilterFilterfilterfilter (input_control) string(-array) → (string)

Downstream filters.

Default: 'none' "none" "none" "none" "none"

List of values: 'left_right_check'"left_right_check""left_right_check""left_right_check""left_right_check", 'none'"none""none""none""none"

SubDistanceSubDistanceSubDistancesubDistancesub_distance (input_control) string(-array) → (string)

Distance interpolation.

Default: 'none' "none" "none" "none" "none"

List of values: 'interpolation'"interpolation""interpolation""interpolation""interpolation", 'none'"none""none""none""none"

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)
* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1, 'viewing_direction', \
                                 'bilinear', CamParamRect1, CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2, RelPoseRect)
* Compute the distance values in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_distance (ImageRect1, ImageRect2, Distance, Score, \
                      CamParamRect1, CamParamRect2, RelPoseRect, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)
* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1, 'viewing_direction', \
                                 'bilinear', CamParamRect1, CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2, RelPoseRect)
* Compute the distance values in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_distance (ImageRect1, ImageRect2, Distance, Score, \
                      CamParamRect1, CamParamRect2, RelPoseRect, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Example (C++)

// Set internal and external stereo parameters.
// Note that, typically, these values are the result of a prior
// calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006,
                                622, 517, 1280, 1024, &CamParam1);
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006,
                                654, 519, 1280, 1024, &CamParam2);
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89,
             "Rp+T", "gba", "point", &RelPose);
// Compute the mapping for rectified images.
gen_binocular_rectification_map(&Map1,&Map2,CamParam1,CamParam2,
                                RelPose,1,"viewing_direction",
                                "bilinear",&CamParamRect1,&CamParamRect2,
                                &CamPoseRect1,&CamPoseRect2,&RelPoseRect);
// Compute the distance values in online images.
while (1)
{
  grab_image_async(&Image1,AcqHandle1,-1);
  map_image(Image1,Map1,&ImageRect1);

  grab_image_async(&Image2,AcqHandle2,-1);
  map_image(Image2,Map2,&ImageRect2);

  binocular_distance(ImageRect1,ImageRect2,&Distance,&Score,
                     CamParamRect1,CamParamRect2,RelPose,"sad",
                     11,11,20,-40,20,2,25,
                     "left_right_check","interpolation");
}

Example (HDevelop)

* Set internal and external stereo parameters.
* Note that, typically, these values are the result of a prior
* calibration.
gen_cam_par_area_scan_division (0.01, -665, 5.2e-006, 5.2e-006, \
                                622, 517, 1280, 1024, CamParam1)
gen_cam_par_area_scan_division (0.01, -731, 5.2e-006, 5.2e-006, \
                                654, 519, 1280, 1024, CamParam2)
create_pose (0.1535,-0.0037,0.0447,0.17,319.84,359.89, \
             'Rp+T', 'gba', 'point', RelPose)
* Compute the mapping for rectified images.
gen_binocular_rectification_map (Map1, Map2, CamParam1, CamParam2, \
                                 RelPose, 1, 'viewing_direction', \
                                 'bilinear', CamParamRect1, CamParamRect2, \
                                 Cam1PoseRect1, Cam2PoseRect2, RelPoseRect)
* Compute the distance values in online images.
while (1)
  grab_image_async (Image1, AcqHandle1, -1)
  map_image (Image1, Map1, ImageRect1)

  grab_image_async (Image2, AcqHandle2, -1)
  map_image (Image2, Map2, ImageRect2)

  binocular_distance (ImageRect1, ImageRect2, Distance, Score, \
                      CamParamRect1, CamParamRect2, RelPoseRect, 'sad', \
                      11, 11, 20, -40, 20, 2, 25, \
                      'left_right_check', 'interpolation')
endwhile

Result

binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity returns 2 ( H_MSG_TRUE) if all parameter values are correct. If necessary, an exception is raised.

Possible Predecessors

map_imagemap_imageMapImageMapImagemap_image

Possible Successors

thresholdthresholdThresholdThresholdthreshold

Alternatives

binocular_distance_mgbinocular_distance_mgBinocularDistanceMgBinocularDistanceMgbinocular_distance_mg, binocular_distance_msbinocular_distance_msBinocularDistanceMsBinocularDistanceMsbinocular_distance_ms, binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity, binocular_disparity_mgbinocular_disparity_mgBinocularDisparityMgBinocularDisparityMgbinocular_disparity_mg, binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsbinocular_disparity_ms

Module

3D Metrology

Operators

Operator Reference

binocular_distanceT_binocular_distanceBinocularDistanceBinocularDistancebinocular_distance (Operator)

Signature

Description

Attention

Execution Information

Parameters

Example (HDevelop)

Example (HDevelop)

Example (C++)

Example (HDevelop)

Result

Possible Predecessors

Possible Successors

Alternatives

See also

Module