Operator Reference

proj_match_points_distortion_ransacT_proj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac (Operator)

proj_match_points_distortion_ransacT_proj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac — Compute a projective transformation matrix between two images and the radial distortion coefficient by automatically finding correspondences between points.

Signature

proj_match_points_distortion_ransac(Image1, Image2 : : Rows1, Cols1, Rows2, Cols2, GrayMatchMethod, MaskSize, RowMove, ColMove, RowTolerance, ColTolerance, Rotation, MatchThreshold, EstimationMethod, DistanceThreshold, RandSeed : HomMat2D, Kappa, Error, Points1, Points2)

Description

Given a set of coordinates of characteristic points (Rows1Rows1Rows1rows1rows_1,Cols1Cols1Cols1cols1cols_1) and (Rows2Rows2Rows2rows2rows_2,Cols2Cols2Cols2cols2cols_2) in both input images Image1Image1Image1image1image_1 and Image2Image2Image2image2image_2, which must be of identical size, proj_match_points_distortion_ransacproj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac automatically determines corresponding points, the homogeneous projective transformation matrix HomMat2DHomMat2DHomMat2DhomMat2Dhom_mat_2d, and the radial distortion coefficient KappaKappaKappakappakappa that optimally fulfill the following equation: Here, and denote image points that are obtained by undistorting the input image points with the division model (see Calibration): Here, and denote the distorted image points, specified relative to the image center, and w and h denote the width and height of the input images. Thus, proj_match_points_distortion_ransacproj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac assumes that the principal point of the camera, i.e., the center of the radial distortions, lies at the center of the image.

The matching process is based on characteristic points, which can be extracted with point operators like points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerpoints_foerstner or points_harrispoints_harrisPointsHarrisPointsHarrispoints_harris. The matching itself is carried out in two steps: first, gray value correlations of mask windows around the input points in the first and the second image are determined and an initial matching between them is generated using the similarity of the windows in both images. Then, the RANSAC algorithm is applied to find the projective transformation matrix and radial distortion coefficient that maximizes the number of correspondences under the above constraint.

The size of the mask windows used for the matching is MaskSizeMaskSizeMaskSizemaskSizemask_size x MaskSizeMaskSizeMaskSizemaskSizemask_size. Three metrics for the correlation can be selected. If GrayMatchMethodGrayMatchMethodGrayMatchMethodgrayMatchMethodgray_match_method has the value 'ssd'"ssd""ssd""ssd""ssd", the sum of the squared gray value differences is used, 'sad'"sad""sad""sad""sad" means the sum of absolute differences, and 'ncc'"ncc""ncc""ncc""ncc" is the normalized cross correlation. For details please refer to binocular_disparitybinocular_disparityBinocularDisparityBinocularDisparitybinocular_disparity. The metric is minimized ('ssd'"ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad") or maximized ('ncc'"ncc""ncc""ncc""ncc") over all possible point pairs. A thus found matching is only accepted if the value of the metric is below the value of MatchThresholdMatchThresholdMatchThresholdmatchThresholdmatch_threshold ('ssd'"ssd""ssd""ssd""ssd", 'sad'"sad""sad""sad""sad") or above that value ('ncc'"ncc""ncc""ncc""ncc").

To increase the algorithm's performance, the search area for the match candidates can be limited to a rectangle by specifying its size and offset. Only points within a window of points are considered. The offset of the center of the search window in the second image with respect to the position of the current point in the first image is given by RowMoveRowMoveRowMoverowMoverow_move and ColMoveColMoveColMovecolMovecol_move.

If the transformation contains a rotation, i.e., if the first image is rotated with respect to the second image, the parameter RotationRotationRotationrotationrotation may contain an estimate for the rotation angle or an angle interval in radians. A good guess will increase the quality of the gray value matching. If the actual rotation differs too much from the specified estimate, the matching will typically fail. In this case, an angle interval should be specified and RotationRotationRotationrotationrotation is a tuple with two elements. The larger the given interval is the slower the operator is since the RANSAC algorithm is run over all (automatically determined) angle increments within the interval.

After the initial matching has been completed, a randomized search algorithm (RANSAC) is used to determine the projective transformation matrix HomMat2DHomMat2DHomMat2DhomMat2Dhom_mat_2d and the radial distortion coefficient KappaKappaKappakappakappa. It tries to find the parameters that are consistent with a maximum number of correspondences. For a point to be accepted, the distance in pixels to its corresponding transformed point must not exceed the threshold DistanceThresholdDistanceThresholdDistanceThresholddistanceThresholddistance_threshold.

The parameter EstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method determines which algorithm is used to compute the projective transformation matrix. A linear algorithm is used if EstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method is set to 'linear'"linear""linear""linear""linear". This algorithm is very fast and returns accurate results for small to moderate noise of the point coordinates and for most distortions (except for small distortions). For EstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard", a mathematically optimal but slower optimization is used, which minimizes the geometric reprojection error. In general, it is preferable to use EstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method = 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard".

The value ErrorErrorErrorerrorerror indicates the overall quality of the estimation procedure and is the mean symmetric euclidean distance in pixels between the points and their corresponding transformed points.

Point pairs consistent with the above constraints are considered to be corresponding points. Points1Points1Points1points1points_1 contains the indices of the matched input points from the first image and Points2Points2Points2points2points_2 contains the indices of the corresponding points in the second image.

The parameter RandSeedRandSeedRandSeedrandSeedrand_seed can be used to control the randomized nature of the RANSAC algorithm, and hence to obtain reproducible results. If RandSeedRandSeedRandSeedrandSeedrand_seed is set to a positive number, the operator returns the same result on every call with the same parameters because the internally used random number generator is initialized with RandSeedRandSeedRandSeedrandSeedrand_seed. If RandSeedRandSeedRandSeedrandSeedrand_seed = 0, the random number generator is initialized with the current time. In this case the results may not be reproducible. The value set for the HALCON system variable 'seed_rand'"seed_rand""seed_rand""seed_rand""seed_rand" (see set_systemset_systemSetSystemSetSystemset_system) does not affect the results of proj_match_points_distortion_ransacproj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac.

Execution Information

Multithreading type: reentrant (runs in parallel with non-exclusive operators).
Multithreading scope: global (may be called from any thread).
Processed without parallelization.

Parameters

Image1Image1Image1image1image_1 (input_object) singlechannelimage → object (byte / uint2)

Input image 1.

Image2Image2Image2image2image_2 (input_object) singlechannelimage → object (byte / uint2)

Input image 2.

Rows1Rows1Rows1rows1rows_1 (input_control) point.y-array → (real / integer)

Input points in image 1 (row coordinate).

Restriction: length(Rows1) >= 5

Cols1Cols1Cols1cols1cols_1 (input_control) point.x-array → (real / integer)

Input points in image 1 (column coordinate).

Restriction: length(Cols1) == length(Rows1)

Rows2Rows2Rows2rows2rows_2 (input_control) point.y-array → (real / integer)

Input points in image 2 (row coordinate).

Restriction: length(Rows2) >= 5

Cols2Cols2Cols2cols2cols_2 (input_control) point.x-array → (real / integer)

Input points in image 2 (column coordinate).

Restriction: length(Cols2) == length(Rows2)

GrayMatchMethodGrayMatchMethodGrayMatchMethodgrayMatchMethodgray_match_method (input_control) string → (string)

Gray value match metric.

Default: 'ncc' "ncc" "ncc" "ncc" "ncc"

List of values: 'ncc'"ncc""ncc""ncc""ncc", 'sad'"sad""sad""sad""sad", 'ssd'"ssd""ssd""ssd""ssd"

MaskSizeMaskSizeMaskSizemaskSizemask_size (input_control) integer → (integer)

Size of gray value masks.

Default: 10

Suggested values: 3, 7, 15

Value range: 1 ≤ MaskSize MaskSize MaskSize maskSize mask_size

RowMoveRowMoveRowMoverowMoverow_move (input_control) integer → (integer)

Average row coordinate offset of corresponding points.

Default: 0

ColMoveColMoveColMovecolMovecol_move (input_control) integer → (integer)

Average column coordinate offset of corresponding points.

Default: 0

RowToleranceRowToleranceRowTolerancerowTolerancerow_tolerance (input_control) integer → (integer)

Half height of matching search window.

Default: 200

Restriction: RowTolerance >= 1

ColToleranceColToleranceColTolerancecolTolerancecol_tolerance (input_control) integer → (integer)

Half width of matching search window.

Default: 200

Restriction: ColTolerance >= 1

RotationRotationRotationrotationrotation (input_control) angle.rad(-array) → (real / integer)

Estimate of the relative rotation of the second image with respect to the first image.

Default: 0.0

Suggested values: 0.0, 0.1, -0.1, 0.7854, 1.571, 3.142

MatchThresholdMatchThresholdMatchThresholdmatchThresholdmatch_threshold (input_control) number → (integer / real)

Threshold for gray value matching.

Default: 0.7

Suggested values: 0.9, 0.7, 0.5, 10, 20, 50, 100

EstimationMethodEstimationMethodEstimationMethodestimationMethodestimation_method (input_control) string → (string)

Algorithm for the computation of the projective transformation matrix.

Default: 'gold_standard' "gold_standard" "gold_standard" "gold_standard" "gold_standard"

List of values: 'gold_standard'"gold_standard""gold_standard""gold_standard""gold_standard", 'linear'"linear""linear""linear""linear"

DistanceThresholdDistanceThresholdDistanceThresholddistanceThresholddistance_threshold (input_control) number → (real / integer)

Threshold for the transformation consistency check.

Default: 1

Restriction: DistanceThreshold > 0

RandSeedRandSeedRandSeedrandSeedrand_seed (input_control) integer → (integer)

Seed for the random number generator.

Default: 0

HomMat2DHomMat2DHomMat2DhomMat2Dhom_mat_2d (output_control) hom_mat2d → (real)

Computed homogeneous projective transformation matrix.

KappaKappaKappakappakappa (output_control) real → (real)

Computed radial distortion coefficient.

ErrorErrorErrorerrorerror (output_control) real → (real)

Root-Mean-Square transformation error.

Points1Points1Points1points1points_1 (output_control) integer-array → (integer)

Indices of matched input points in image 1.

Points2Points2Points2points2points_2 (output_control) integer-array → (integer)

Indices of matched input points in image 2.

Example (HDevelop)

points_foerstner (Image1, 1, 2, 3, 50, 0.1, 'gauss', 'true', \
                  Rows1, Cols1, _, _, _, _, _, _, _, _)
points_foerstner (Image2, 1, 2, 3, 50, 0.1, 'gauss', 'true', \
                  Rows2, Cols2, _, _, _, _, _, _, _, _)
get_image_size (Image1, Width, Height)
proj_match_points_distortion_ransac (Image1, Image2, Rows1, Cols1, \
                                     Rows2, Cols2, 'ncc', 10, 0, 0, \
                                     Height, Width, 0, 0.5, \
                                     'gold_standard', 1, 42, \
                                     HomMat2D, Kappa, Error, \
                                     Points1, Points2)
CamParDist := ['area_scan_division',0.0,Kappa,1.0,1.0, \
               0.5*(Width-1),0.5*(Height-1),Width,Height]
change_radial_distortion_cam_par ('fixed', CamParDist, 0, CamPar)
change_radial_distortion_image (Image1, Image1, Image1Rect, \
                                CamParDist, CamPar)
change_radial_distortion_image (Image2, Image2, Image2Rect, \
                                CamParDist, CamPar)
concat_obj (Image1Rect, Image2Rect, ImagesRect)
gen_projective_mosaic (ImagesRect, MosaicImage, 1, 1, 2, HomMat2D, \
                       'default', 'false', MosaicMatrices2D)

Possible Predecessors

points_foerstnerpoints_foerstnerPointsFoerstnerPointsFoerstnerpoints_foerstner, points_harrispoints_harrisPointsHarrisPointsHarrispoints_harris

Possible Successors

Alternatives

proj_match_points_distortion_ransac_guidedproj_match_points_distortion_ransac_guidedProjMatchPointsDistortionRansacGuidedProjMatchPointsDistortionRansacGuidedproj_match_points_distortion_ransac_guided

References

Richard Hartley, Andrew Zisserman: “Multiple View Geometry in Computer Vision”; Cambridge University Press, Cambridge; 2003.
Olivier Faugeras, Quang-Tuan Luong: “The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applications”; MIT Press, Cambridge, MA; 2001.

Module

Matching

Operators

Operator Reference

proj_match_points_distortion_ransacT_proj_match_points_distortion_ransacProjMatchPointsDistortionRansacProjMatchPointsDistortionRansacproj_match_points_distortion_ransac (Operator)

Signature

Description

Execution Information

Parameters

Example (HDevelop)

Possible Predecessors

Possible Successors

Alternatives

See also

References

Module