Skip to content

binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsbinocular_disparity_msbinocular_disparity_ms🔗

Short description🔗

binocular_disparity_msbinocular_disparity_msBinocularDisparityMsBinocularDisparityMsbinocular_disparity_msbinocular_disparity_ms — Compute the disparities of a rectified stereo image pair using multi-scanline optimization.

Signature🔗

binocular_disparity_ms( image ImageRect1, image ImageRect2, out image Disparity, out image Score, integer MinDisparity, integer MaxDisparity, integer SurfaceSmoothing, integer EdgeSmoothing, attribute.name GenParamName, attribute.value GenParamValue )binocular_disparity_ms( ImageRect1, ImageRect2 : Disparity, Score : MinDisparity, MaxDisparity, SurfaceSmoothing, EdgeSmoothing, GenParamName, GenParamValue : )void BinocularDisparityMs( const HObject& ImageRect1, const HObject& ImageRect2, HObject* Disparity, HObject* Score, const HTuple& MinDisparity, const HTuple& MaxDisparity, const HTuple& SurfaceSmoothing, const HTuple& EdgeSmoothing, const HTuple& GenParamName, const HTuple& GenParamValue )static void HOperatorSet.BinocularDisparityMs( HObject imageRect1, HObject imageRect2, out HObject disparity, out HObject score, HTuple minDisparity, HTuple maxDisparity, HTuple surfaceSmoothing, HTuple edgeSmoothing, HTuple genParamName, HTuple genParamValue )def binocular_disparity_ms( image_rect_1: HObject, image_rect_2: HObject, min_disparity: int, max_disparity: int, surface_smoothing: int, edge_smoothing: int, gen_param_name: MaybeSequence[str], gen_param_value: MaybeSequence[str] ) -> Tuple[HObject, HObject]

Herror binocular_disparity_ms( const Hobject ImageRect1, const Hobject ImageRect2, Hobject* Disparity, Hobject* Score, const Hlong MinDisparity, const Hlong MaxDisparity, const Hlong SurfaceSmoothing, const Hlong EdgeSmoothing, const char* GenParamName, const char* GenParamValue )

Herror T_binocular_disparity_ms( const Hobject ImageRect1, const Hobject ImageRect2, Hobject* Disparity, Hobject* Score, const Htuple MinDisparity, const Htuple MaxDisparity, const Htuple SurfaceSmoothing, const Htuple EdgeSmoothing, const Htuple GenParamName, const Htuple GenParamValue )

HImage HImage::BinocularDisparityMs( const HImage& ImageRect2, HImage* Score, Hlong MinDisparity, Hlong MaxDisparity, Hlong SurfaceSmoothing, Hlong EdgeSmoothing, const HTuple& GenParamName, const HTuple& GenParamValue ) const

HImage HImage::BinocularDisparityMs( const HImage& ImageRect2, HImage* Score, Hlong MinDisparity, Hlong MaxDisparity, Hlong SurfaceSmoothing, Hlong EdgeSmoothing, const HString& GenParamName, const HString& GenParamValue ) const

HImage HImage::BinocularDisparityMs( const HImage& ImageRect2, HImage* Score, Hlong MinDisparity, Hlong MaxDisparity, Hlong SurfaceSmoothing, Hlong EdgeSmoothing, const char* GenParamName, const char* GenParamValue ) const

HImage HImage::BinocularDisparityMs( const HImage& ImageRect2, HImage* Score, Hlong MinDisparity, Hlong MaxDisparity, Hlong SurfaceSmoothing, Hlong EdgeSmoothing, const wchar_t* GenParamName, const wchar_t* GenParamValue ) const (Windows only)

HImage HImage.BinocularDisparityMs( HImage imageRect2, out HImage score, int minDisparity, int maxDisparity, int surfaceSmoothing, int edgeSmoothing, HTuple genParamName, HTuple genParamValue )

HImage HImage.BinocularDisparityMs( HImage imageRect2, out HImage score, int minDisparity, int maxDisparity, int surfaceSmoothing, int edgeSmoothing, string genParamName, string genParamValue )

Description🔗

binocular_disparity_msBinocularDisparityMs calculates the disparity between two rectified stereo images ImageRect1imageRect1image_rect_1 and ImageRect2imageRect2image_rect_2 using multi-scanline optimization. The resulting disparity image is returned in Disparitydisparitydisparity. In contrast to binocular_distance_msBinocularDistanceMs, the results are not transformed into distance values.

For this task, the three operators binocular_disparityBinocularDisparity, binocular_disparity_mgBinocularDisparityMg, and binocular_disparity_msBinocularDisparityMs can be used. binocular_disparityBinocularDisparity returns robust results in regions of sufficient texture but fails where there is none. binocular_disparity_mgBinocularDisparityMg interpolates low-texture regions but blurs discontinuities. binocular_disparity_msBinocularDisparityMs preserves discontinuities and interpolates partially.

binocular_disparity_msBinocularDisparityMs requires a reference image ImageRect1imageRect1image_rect_1 and a search image ImageRect2imageRect2image_rect_2 which both must be rectified, i.e., corresponding pixels must have the same row coordinate. If this assumption is violated, the images can be rectified by using the operators calibrate_camerasCalibrateCameras, gen_binocular_rectification_mapGenBinocularRectificationMap, and map_imageMapImage.

ImageRect1imageRect1image_rect_1 and ImageRect2imageRect2image_rect_2 can have different widths but must have the same height. Given a pixel in ImageRect1imageRect1image_rect_1, the homologous pixel in ImageRect2imageRect2image_rect_2 is selected by searching along the corresponding row in ImageRect2imageRect2image_rect_2 and matching both pixels based on a similarity measure. The disparity is the number of pixels by which each pixel in ImageRect1imageRect1image_rect_1 needs to be moved to reach the homologous pixel in ImageRect2imageRect2image_rect_2.

The search space is confined by the minimum and maximum disparity values MinDisparityminDisparitymin_disparity and MaxDisparitymaxDisparitymax_disparity. If the minimum and maximum disparity values are set to an empty tuple, they are automatically set to the maximum possible range for the given images ImageRect1imageRect1image_rect_1.

To calculate the disparities from the similarity measure, the intermediate results are optimized by a multi-scanline method. The optimization increases the robustness in low-texture areas without blurring discontinuities in the disparity image. The optimization is controlled by the parameters SurfaceSmoothingsurfaceSmoothingsurface_smoothing and EdgeSmoothingedgeSmoothingedge_smoothing. SurfaceSmoothingsurfaceSmoothingsurface_smoothing controls the smoothness within surfaces. High values suppress disparity differences of one pixel. EdgeSmoothingedgeSmoothingedge_smoothing controls the occurrence and the shape of edges. Low values allow many edges, high values lead to fewer and rounder edges. For both parameters, reasonable values usually range between 0 and 100. If both parameters are set to zero, no optimization is performed.

The calculation of the disparities can be controlled by generic parameters. The following generic parameters GenParamNamegenParamNamegen_param_name and the corresponding values GenParamValuegenParamValuegen_param_value are supported:

  • 'consistency_check'"consistency_check" Activates an optional post-processing step to increase robustness. Concurrent direct and reverse matches between reference patterns in ImageRect1imageRect1image_rect_1 and ImageRect2imageRect2image_rect_2 are required for a disparity value to be returned. The check is switched off by setting GenParamValuegenParamValuegen_param_value to 'false'"false".

    List of values: 'true'"true", 'false'"false".

    Default: 'true'"true".

  • 'disparity_offset'"disparity_offset" Adapts the quality of the coarse-to-fine approach at discontinuities. The higher the value set in GenParamValuegenParamValuegen_param_value, the more runtime is required.

    Suggested values: 22, 33, 44.

    Default: 33.

  • 'method'"method": Determines the method used to calculate the disparities. The following parameters GenParamValuegenParamValuegen_param_value can be set:

    • 'accurate'"accurate": Most accurate calculation method, but requires more runtime and memory compared to the remaining methods.

    • 'fast'"fast": Uses a coarse-to-fine scheme to improve the runtime. The coarse-to-fine scheme works in a similar way to the scheme explained in binocular_disparityBinocularDisparity.

      The coarse-to-fine method requires significantly less memory and is significantly faster than the 'accurate'"accurate" method, especially for large images or a large range of MinDisparityminDisparitymin_disparity and MaxDisparitymaxDisparitymax_disparity. The coarse-to-fine scheme has the further advantage that it automatically estimates the range of MinDisparityminDisparitymin_disparity and MaxDisparitymaxDisparitymax_disparity while traversing through the pyramid. As a consequence, neither MinDisparityminDisparitymin_disparity nor MaxDisparitymaxDisparitymax_disparity needs to be set. However, the generated disparity images are less accurate for the 'fast'"fast" method than for the default 'accurate'"accurate" approach. Especially at sharp disparity jumps the 'fast'"fast" method preserves discontinuities less accurately.

    • 'very_fast'"very_fast": Also uses a coarse-to-fine scheme to improve the runtime even further. However, this approach makes numerous assumptions that may lead to a smoothing of the disparities at discontinuities. Per default, the number of levels of the coarse-to-fine scheme is estimated automatically. It is possible to set the number of levels explicitly (see 'num_levels'"num_levels").

    The runtime of the operator is approximately linear to the image width, the image height, and the disparity range. Consequently, the disparity range should be chosen as narrow as possible for large images. The runtime of the coarse-to-fine scheme (which is used for 'fast'"fast" or 'very_fast'"very_fast") is approximately linear to the image width and the image height. For small images and small disparity ranges the runtime of the coarse-to-fine scheme may be larger than that of the 'accurate'"accurate" scheme.

    List of values: 'accurate'"accurate", 'fast'"fast", 'very_fast'"very_fast".

    Default: 'accurate'"accurate".

  • 'num_levels'"num_levels": Determines the number of pyramid levels that are used for the coarse-to-fine scheme. By setting GenParamValuegenParamValuegen_param_value to 'auto'"auto", the number of pyramid levels is automatically calculated.

    Suggested values: 22, 33, 'auto'"auto".

    Default: 'auto'"auto".

  • 'similarity_measure'"similarity_measure": Sets the similarity measure to be used. For both options 'census_dense'"census_dense" (default) and 'census_sparse'"census_sparse", the similarity measure is based on the Census transform. A Census transformed image contains for every pixel information about the intensity topology within a support window around it.

    • 'census_dense'"census_dense": Uses a dense 9 x 7 pixels window and is more suitable for fine structures.

    • 'census_sparse'"census_sparse": Uses a sparse 15 x 15 pixels window where only a subset of the pixels is evaluated. Is more robust in low-texture areas.

    List of values: 'census_dense'"census_dense", 'census_sparse'"census_sparse".

    Default: 'census_dense'"census_dense".

  • 'sub_disparity'"sub_disparity": Activates sub-pixel refinement of disparities when set to 'true'"true". Can be deactivated by setting 'false'"false".

    List of values: 'true'"true", 'false'"false".

    Default: 'true'"true".

The resulting disparity is returned in the single-channel image Disparitydisparitydisparity. A quality measure for each disparity value is returned in Scorescorescore, containing the best (lowest) result of the optimized similarity measure of a reference pixel.

Execution information🔗

Execution information
  • Supports OpenCL compute devices.

  • Multithreading type: reentrant (runs in parallel with non-exclusive operators).

  • Multithreading scope: global (may be called from any thread).

  • Automatically parallelized on internal data level.

Parameters🔗

ImageRect1imageRect1image_rect_1 (input_object) singlechannelimage → object (byte*)object (byte*)HObject (byte*)HImage (byte*)HObject (byte*)Hobject (byte*) *allowed for compute devices

Rectified image of camera 1.

ImageRect2imageRect2image_rect_2 (input_object) singlechannelimage → object (byte*)object (byte*)HObject (byte*)HImage (byte*)HObject (byte*)Hobject (byte*) *allowed for compute devices

Rectified image of camera 2.

Disparitydisparitydisparity (output_object) singlechannelimage → object (real)object (real)HObject (real)HImage (real)HObject (real)Hobject * (real)

Disparity map.

Scorescorescore (output_object) singlechannelimage → object (real)object (real)HObject (real)HImage (real)HObject (real)Hobject * (real)

Score of the calculated disparity.

MinDisparityminDisparitymin_disparity (input_control) integer → (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Minimum of the expected disparities.

Default: -30-30
Value range: -32768 ≤ MinDisparity ≤ 32768
Restriction: MinDisparity <= MaxDisparity

MaxDisparitymaxDisparitymax_disparity (input_control) integer → (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Maximum of the expected disparities.

Default: 3030
Value range: -32768 ≤ MaxDisparity ≤ 32768
Restriction: MinDisparity <= MaxDisparity

SurfaceSmoothingsurfaceSmoothingsurface_smoothing (input_control) integer → (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Smoothing of surfaces.

Default: 5050
Suggested values: 20, 50, 10020, 50, 100
Restriction: SurfaceSmoothing >= 0

EdgeSmoothingedgeSmoothingedge_smoothing (input_control) integer → (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Smoothing of edges.

Default: 5050
Suggested values: 20, 50, 10020, 50, 100
Restriction: EdgeSmoothing >= 0

GenParamNamegenParamNamegen_param_name (input_control) attribute.name(-array) → (string)(string)HTuple (HString)HTuple (string)MaybeSequence[str]Htuple (char*)

Parameter name(s) for the multi-scanline algorithm.

Default: [][]
List of values: 'consistency_check', 'disparity_offset', 'method', 'num_levels', 'similarity_measure', 'sub_disparity'"consistency_check", "disparity_offset", "method", "num_levels", "similarity_measure", "sub_disparity"

GenParamValuegenParamValuegen_param_value (input_control) attribute.value(-array) → (string)(string)HTuple (HString)HTuple (string)MaybeSequence[str]Htuple (char*)

Parameter value(s) for the multi-scanline algorithm.

Default: [][]
Suggested values: 'accurate', 'fast', 'very_fast', 'census_dense', 'census_sparse', 'true', 'false', 'auto'"accurate", "fast", "very_fast", "census_dense", "census_sparse", "true", "false", "auto"

Example🔗

(HDevelop)


read_image (BaseballL, 'stereo/epipolar/baseball_l')
read_image (BaseballR, 'stereo/epipolar/baseball_r')
binocular_disparity_ms (BaseballL, BaseballR, Disparity, Score, \
                        -40, -10, 50, 50, [], [])

Result🔗

If the parameter values are correct, binocular_disparity_msBinocularDisparityMs returns the value 2 (H_MSG_TRUE). If the input is empty (no input images are available) the behavior can be set via set_system('no_object_result',<Result>). If necessary, an exception is raised.

Combinations with other operators🔗

Combinations

Possible predecessors

map_imageMapImage

Possible successors

thresholdThreshold, disparity_to_distanceDisparityToDistance, disparity_image_to_xyzDisparityImageToXyz

Alternatives

binocular_disparityBinocularDisparity, binocular_disparity_mgBinocularDisparityMg, binocular_distanceBinocularDistance, binocular_distance_mgBinocularDistanceMg, binocular_distance_msBinocularDistanceMs

See also

map_imageMapImage, gen_binocular_rectification_mapGenBinocularRectificationMap, binocular_calibrationBinocularCalibration

Module🔗

3D Metrology