Skip to content

match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransacT_match_fundamental_matrix_ransacπŸ”—

Short descriptionπŸ”—

match_fundamental_matrix_ransacmatch_fundamental_matrix_ransacMatchFundamentalMatrixRansacMatchFundamentalMatrixRansacmatch_fundamental_matrix_ransacT_match_fundamental_matrix_ransac β€” Compute the fundamental matrix for a pair of stereo images by automatically finding correspondences between image points.

SignatureπŸ”—

match_fundamental_matrix_ransac( image Image1, image Image2, number Rows1, number Cols1, number Rows2, number Cols2, string GrayMatchMethod, integer MaskSize, integer RowMove, integer ColMove, integer RowTolerance, integer ColTolerance, angle.rad Rotation, number MatchThreshold, string EstimationMethod, number DistanceThreshold, integer RandSeed, out hom_mat2d FMatrix, out real CovFMat, out real Error, out integer Points1, out integer Points2 )match_fundamental_matrix_ransac( Image1, Image2 : : Rows1, Cols1, Rows2, Cols2, GrayMatchMethod, MaskSize, RowMove, ColMove, RowTolerance, ColTolerance, Rotation, MatchThreshold, EstimationMethod, DistanceThreshold, RandSeed : FMatrix, CovFMat, Error, Points1, Points2 )void MatchFundamentalMatrixRansac( const HObject& Image1, const HObject& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const HTuple& GrayMatchMethod, const HTuple& MaskSize, const HTuple& RowMove, const HTuple& ColMove, const HTuple& RowTolerance, const HTuple& ColTolerance, const HTuple& Rotation, const HTuple& MatchThreshold, const HTuple& EstimationMethod, const HTuple& DistanceThreshold, const HTuple& RandSeed, HTuple* FMatrix, HTuple* CovFMat, HTuple* Error, HTuple* Points1, HTuple* Points2 )static void HOperatorSet.MatchFundamentalMatrixRansac( HObject image1, HObject image2, HTuple rows1, HTuple cols1, HTuple rows2, HTuple cols2, HTuple grayMatchMethod, HTuple maskSize, HTuple rowMove, HTuple colMove, HTuple rowTolerance, HTuple colTolerance, HTuple rotation, HTuple matchThreshold, HTuple estimationMethod, HTuple distanceThreshold, HTuple randSeed, out HTuple FMatrix, out HTuple covFMat, out HTuple error, out HTuple points1, out HTuple points2 )def match_fundamental_matrix_ransac( image_1: HObject, image_2: HObject, rows_1: Sequence[Union[float, int]], cols_1: Sequence[Union[float, int]], rows_2: Sequence[Union[float, int]], cols_2: Sequence[Union[float, int]], gray_match_method: str, mask_size: int, row_move: int, col_move: int, row_tolerance: int, col_tolerance: int, rotation: MaybeSequence[Union[float, int]], match_threshold: Union[int, float], estimation_method: str, distance_threshold: Union[float, int], rand_seed: int ) -> Tuple[Sequence[float], Sequence[float], float, Sequence[int], Sequence[int]]

Herror T_match_fundamental_matrix_ransac( const Hobject Image1, const Hobject Image2, const Htuple Rows1, const Htuple Cols1, const Htuple Rows2, const Htuple Cols2, const Htuple GrayMatchMethod, const Htuple MaskSize, const Htuple RowMove, const Htuple ColMove, const Htuple RowTolerance, const Htuple ColTolerance, const Htuple Rotation, const Htuple MatchThreshold, const Htuple EstimationMethod, const Htuple DistanceThreshold, const Htuple RandSeed, Htuple* FMatrix, Htuple* CovFMat, Htuple* Error, Htuple* Points1, Htuple* Points2 )

HHomMat2D HImage::MatchFundamentalMatrixRansac( const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const HString& GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, const HTuple& Rotation, const HTuple& MatchThreshold, const HString& EstimationMethod, const HTuple& DistanceThreshold, Hlong RandSeed, HTuple* CovFMat, double* Error, HTuple* Points1, HTuple* Points2 ) const

HHomMat2D HImage::MatchFundamentalMatrixRansac( const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const HString& GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const HString& EstimationMethod, double DistanceThreshold, Hlong RandSeed, HTuple* CovFMat, double* Error, HTuple* Points1, HTuple* Points2 ) const

HHomMat2D HImage::MatchFundamentalMatrixRansac( const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const char* GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const char* EstimationMethod, double DistanceThreshold, Hlong RandSeed, HTuple* CovFMat, double* Error, HTuple* Points1, HTuple* Points2 ) const

HHomMat2D HImage::MatchFundamentalMatrixRansac( const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const wchar_t* GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const wchar_t* EstimationMethod, double DistanceThreshold, Hlong RandSeed, HTuple* CovFMat, double* Error, HTuple* Points1, HTuple* Points2 ) const (Windows only)

HTuple HHomMat2D::MatchFundamentalMatrixRansac( const HImage& Image1, const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const HString& GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, const HTuple& Rotation, const HTuple& MatchThreshold, const HString& EstimationMethod, const HTuple& DistanceThreshold, Hlong RandSeed, double* Error, HTuple* Points1, HTuple* Points2 )

HTuple HHomMat2D::MatchFundamentalMatrixRansac( const HImage& Image1, const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const HString& GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const HString& EstimationMethod, double DistanceThreshold, Hlong RandSeed, double* Error, HTuple* Points1, HTuple* Points2 )

HTuple HHomMat2D::MatchFundamentalMatrixRansac( const HImage& Image1, const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const char* GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const char* EstimationMethod, double DistanceThreshold, Hlong RandSeed, double* Error, HTuple* Points1, HTuple* Points2 )

HTuple HHomMat2D::MatchFundamentalMatrixRansac( const HImage& Image1, const HImage& Image2, const HTuple& Rows1, const HTuple& Cols1, const HTuple& Rows2, const HTuple& Cols2, const wchar_t* GrayMatchMethod, Hlong MaskSize, Hlong RowMove, Hlong ColMove, Hlong RowTolerance, Hlong ColTolerance, double Rotation, Hlong MatchThreshold, const wchar_t* EstimationMethod, double DistanceThreshold, Hlong RandSeed, double* Error, HTuple* Points1, HTuple* Points2 ) (Windows only)

HHomMat2D HImage.MatchFundamentalMatrixRansac( HImage image2, HTuple rows1, HTuple cols1, HTuple rows2, HTuple cols2, string grayMatchMethod, int maskSize, int rowMove, int colMove, int rowTolerance, int colTolerance, HTuple rotation, HTuple matchThreshold, string estimationMethod, HTuple distanceThreshold, int randSeed, out HTuple covFMat, out double error, out HTuple points1, out HTuple points2 )

HHomMat2D HImage.MatchFundamentalMatrixRansac( HImage image2, HTuple rows1, HTuple cols1, HTuple rows2, HTuple cols2, string grayMatchMethod, int maskSize, int rowMove, int colMove, int rowTolerance, int colTolerance, double rotation, int matchThreshold, string estimationMethod, double distanceThreshold, int randSeed, out HTuple covFMat, out double error, out HTuple points1, out HTuple points2 )

HTuple HHomMat2D.MatchFundamentalMatrixRansac( HImage image1, HImage image2, HTuple rows1, HTuple cols1, HTuple rows2, HTuple cols2, string grayMatchMethod, int maskSize, int rowMove, int colMove, int rowTolerance, int colTolerance, HTuple rotation, HTuple matchThreshold, string estimationMethod, HTuple distanceThreshold, int randSeed, out double error, out HTuple points1, out HTuple points2 )

HTuple HHomMat2D.MatchFundamentalMatrixRansac( HImage image1, HImage image2, HTuple rows1, HTuple cols1, HTuple rows2, HTuple cols2, string grayMatchMethod, int maskSize, int rowMove, int colMove, int rowTolerance, int colTolerance, double rotation, int matchThreshold, string estimationMethod, double distanceThreshold, int randSeed, out double error, out HTuple points1, out HTuple points2 )

DescriptionπŸ”—

Given a set of coordinates of characteristic points \((\textrm{Rows1},\textrm{Cols1})\) and \((\textrm{Rows2},\textrm{Cols2})\) in the stereo images Image1image1image_1 and Image2image2image_2, match_fundamental_matrix_ransacMatchFundamentalMatrixRansac automatically finds the correspondences between the characteristic points and determines the geometry of the stereo setup. For unknown cameras the geometry of the stereo setup is represented by the fundamental matrix FMatrixFMatrixfmatrix and all corresponding points have to fulfill the epipolar constraint, namely:

\[\begin{eqnarray*} \left(\begin{array}{c} \textrm{Cols2} \\ \textrm{Rows2} \\ 1 \end{array} \right)^T \cdot \textrm{FMatrix} \cdot \left(\begin{array}{c} \textrm{Cols1} \\ \textrm{Rows1} \\ 1 \end{array} \right) = 0 \quad . \end{eqnarray*}\]

Note the column/row ordering in the point coordinates: because the fundamental matrix encodes the projective relation between two stereo images embedded in 3D space, the x/y notation has to be compliant with the camera coordinate system. So, (x,y) coordinates correspond to (column,row) pairs.

The matching process is based on characteristic points, which can be extracted with point operators like points_foerstnerPointsFoerstner or points_harrisPointsHarris. The matching itself is carried out in two steps: first, gray value correlations of mask windows around the input points in the first and the second image are determined and an initial matching between them is generated using the similarity of the windows in both images. Then, the RANSAC algorithm is applied to find the fundamental matrix that maximizes the number of correspondences under the epipolar constraint.

The size of the mask windows is MaskSize x MaskSize. Three metrics for the correlation can be selected. If GrayMatchMethodgrayMatchMethodgray_match_method has the value 'ssd'"ssd", the sum of the squared gray value differences is used, 'sad'"sad" means the sum of absolute differences, and 'ncc'"ncc" is the normalized cross correlation. For details please refer to binocular_disparityBinocularDisparity. The metric is minimized ('ssd'"ssd", 'sad'"sad") or maximized ('ncc'"ncc") over all possible point pairs. A thus found matching is only accepted if the value of the metric is below the value of MatchThresholdmatchThresholdmatch_threshold ('ssd'"ssd", 'sad'"sad") or above that value ('ncc'"ncc").

To increase the speed of the algorithm the search area for the matching operations can be limited. Only points within a window of

\[\begin{eqnarray*}2 \cdot \textrm{RowTolerance} \times 2 \cdot \textrm{ColTolerance}\end{eqnarray*}\]

points are considered. The offset of the center of the search window in the second image with respect to the position of the current point in the first image is given by RowMoverowMoverow_move and ColMovecolMovecol_move.

If the second camera is rotated around the optical axis with respect to the first camera the parameter Rotationrotationrotation may contain an estimate for the rotation angle or an angle interval in radians. A good guess will increase the quality of the gray value matching. If the actual rotation differs too much from the specified estimate the matching will typically fail. In this case, an angle interval should be specified and Rotationrotationrotation is a tuple with two elements. The larger the given interval the slower the operator is since the RANSAC algorithm is run over all angle increments within the interval.

After the initial matching is completed a randomized search algorithm (RANSAC) is used to determine the fundamental matrix FMatrixFMatrixfmatrix. It tries to find the matrix that is consistent with a maximum number of correspondences. For a point to be accepted, the distance to its corresponding epipolar line must not exceed the threshold DistanceThresholddistanceThresholddistance_threshold.

The parameter EstimationMethodestimationMethodestimation_method decides whether the relative orientation between the cameras is of a special type and which algorithm is to be applied for its computation. If EstimationMethodestimationMethodestimation_method is either 'normalized_dlt'"normalized_dlt" or 'gold_standard'"gold_standard" the relative orientation is arbitrary. If left and right camera are identical and the relative orientation between them is a pure translation then choose EstimationMethodestimationMethodestimation_method equal to 'trans_normalized_dlt'"trans_normalized_dlt" or 'trans_gold_standard'"trans_gold_standard". The typical application for this special motion case is the scenario of a single fixed camera looking onto a moving conveyor belt. In order to get a unique solution in the correspondence problem the minimum required number of corresponding points is eight in the general case and three in the special, translational case.

The fundamental matrix is computed by a linear algorithm if 'normalized_dlt'"normalized_dlt" or 'trans_normalized_dlt'"trans_normalized_dlt" is chosen. With 'gold_standard'"gold_standard" or 'trans_gold_standard'"trans_gold_standard" the algorithm gives a statistically optimal result, and returns as well the covariance of the fundamental matrix CovFMatcovFMatcov_fmat. Here, 'normalized_dlt'"normalized_dlt" and 'gold_standard'"gold_standard" stand for direct-linear-transformation and gold-standard-algorithm respectively.

The value Errorerrorerror indicates the overall quality of the estimation procedure and is the mean Euclidean distance in pixels between the points and their corresponding epipolar lines.

Point pairs consistent with the mentioned constraints are considered to be in correspondences. Points1points1points_1 contains the indices of the matched input points from the first image and Points2points2points_2 contains the indices of the corresponding points in the second image.

The parameter RandSeedrandSeedrand_seed can be used to control the randomized nature of the RANSAC algorithm, and hence to obtain reproducible results. If RandSeedrandSeedrand_seed is set to a positive number the operator yields the same result on every call with the same parameters because the internally used random number generator is initialized with the RandSeedrandSeedrand_seed. If RandSeedrandSeedrand_seed \(=\) 00 the random number generator is initialized with the current time. In this case the results may not be reproducible. The value set for the HALCON system variable 'seed_rand'"seed_rand" (see set_systemSetSystem) does not affect the results of match_fundamental_matrix_ransacMatchFundamentalMatrixRansac.

Execution informationπŸ”—

Execution information
  • Multithreading type: reentrant (runs in parallel with non-exclusive operators).

  • Multithreading scope: global (may be called from any thread).

  • Processed without parallelization.

ParametersπŸ”—

Image1image1image_1 (input_object) singlechannelimage β†’ object (byte / uint2)object (byte / uint2)HObject (byte / uint2)HImage (byte / uint2)HObject (byte / uint2)Hobject (byte / uint2)

Input image 1.

Image2image2image_2 (input_object) singlechannelimage β†’ object (byte / uint2)object (byte / uint2)HObject (byte / uint2)HImage (byte / uint2)HObject (byte / uint2)Hobject (byte / uint2)

Input image 2.

Rows1rows1rows_1 (input_control) number-array β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Row coordinates of characteristic points in image 1.

Restriction: length(Rows1) >= 8 || length(Rows1) >= 3

Cols1cols1cols_1 (input_control) number-array β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Column coordinates of characteristic points in image 1.

Restriction: length(Cols1) == length(Rows1)

Rows2rows2rows_2 (input_control) number-array β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Row coordinates of characteristic points in image 2.

Restriction: length(Rows2) >= 8 || length(Rows2) >= 3

Cols2cols2cols_2 (input_control) number-array β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)Sequence[Union[float, int]]Htuple (double / Hlong)

Column coordinates of characteristic points in image 2.

Restriction: length(Cols2) == length(Rows2)

GrayMatchMethodgrayMatchMethodgray_match_method (input_control) string β†’ (string)(string)HTuple (HString)HTuple (string)strHtuple (char*)

Gray value comparison metric.

Default: 'ssd'"ssd"
List of values: 'ncc', 'sad', 'ssd'"ncc", "sad", "ssd"

MaskSizemaskSizemask_size (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Size of gray value masks.

Default: 1010
Suggested values: 3, 7, 153, 7, 15
Value range: 1 ≀ MaskSize

RowMoverowMoverow_move (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Average row coordinate shift of corresponding points.

Default: 00
Value range: 0 ≀ RowMove ≀ 200

ColMovecolMovecol_move (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Average column coordinate shift of corresponding points.

Default: 00
Value range: 0 ≀ ColMove ≀ 200

RowTolerancerowTolerancerow_tolerance (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Half height of matching search window.

Default: 200200
Value range: 1 ≀ RowTolerance

ColTolerancecolTolerancecol_tolerance (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Half width of matching search window.

Default: 200200
Value range: 1 ≀ ColTolerance

Rotationrotationrotation (input_control) angle.rad(-array) β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)MaybeSequence[Union[float, int]]Htuple (double / Hlong)

Estimate of the relative orientation of the right image with respect to the left image.

Default: 0.00.0
Suggested values: 0.0, 0.1, -0.1, 0.7854, 1.571, 3.1420.0, 0.1, -0.1, 0.7854, 1.571, 3.142

MatchThresholdmatchThresholdmatch_threshold (input_control) number β†’ (integer / real)(integer / real)HTuple (Hlong / double)HTuple (int / long / double)Union[int, float]Htuple (Hlong / double)

Threshold for gray value matching.

Default: 1010
Suggested values: 10, 20, 50, 100, 0.9, 0.710, 20, 50, 100, 0.9, 0.7

EstimationMethodestimationMethodestimation_method (input_control) string β†’ (string)(string)HTuple (HString)HTuple (string)strHtuple (char*)

Algorithm for the computation of the fundamental matrix and for special camera orientations.

Default: 'normalized_dlt'"normalized_dlt"
List of values: 'gold_standard', 'normalized_dlt', 'trans_gold_standard', 'trans_normalized_dlt'"gold_standard", "normalized_dlt", "trans_gold_standard", "trans_normalized_dlt"

DistanceThresholddistanceThresholddistance_threshold (input_control) number β†’ (real / integer)(real / integer)HTuple (double / Hlong)HTuple (double / int / long)Union[float, int]Htuple (double / Hlong)

Maximal deviation of a point from its epipolar line.

Default: 11
Value range: 0.5 ≀ DistanceThreshold ≀ 5
Restriction: DistanceThreshold > 0

RandSeedrandSeedrand_seed (input_control) integer β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Seed for the random number generator.

Default: 00

FMatrixFMatrixfmatrix (output_control) hom_mat2d β†’ (real)(real)HTuple (double)HHomMat2D, HTuple (double)Sequence[float]Htuple (double)

Computed fundamental matrix.

CovFMatcovFMatcov_fmat (output_control) real-array β†’ (real)(real)HTuple (double)HTuple (double)Sequence[float]Htuple (double)

9x9 covariance matrix of the fundamental matrix.

Errorerrorerror (output_control) real β†’ (real)(real)HTuple (double)HTuple (double)floatHtuple (double)

Root-Mean-Square of the epipolar distance error.

Points1points1points_1 (output_control) integer-array β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)Sequence[int]Htuple (Hlong)

Indices of matched input points in image 1.

Points2points2points_2 (output_control) integer-array β†’ (integer)(integer)HTuple (Hlong)HTuple (int / long)Sequence[int]Htuple (Hlong)

Indices of matched input points in image 2.

Combinations with other operatorsπŸ”—

Combinations

Possible predecessors

points_foerstnerPointsFoerstner, points_harrisPointsHarris

Possible successors

vector_to_fundamental_matrixVectorToFundamentalMatrix, gen_binocular_proj_rectificationGenBinocularProjRectification

See also

match_essential_matrix_ransacMatchEssentialMatrixRansac, match_rel_pose_ransacMatchRelPoseRansac, proj_match_points_ransacProjMatchPointsRansac

ReferencesπŸ”—

Richard Hartley, Andrew Zisserman: ``Multiple View Geometry in Computer Visionβ€™β€˜; Cambridge University Press, Cambridge; 2003.

Olivier Faugeras, Quang-Tuan Luong: ``The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applicationsβ€™β€˜; MIT Press, Cambridge, MA; 2001.

ModuleπŸ”—

3D Metrology