Optimizationπ
To locate the same interest points that are stored and described in the model in unknown images of the same or a similar object, the following operators are applied:
find_calib_descriptor_modelsearches for the best matches of a calibrated descriptor model. It returns the 3D pose of the object and the score describing the quality of the match.find_uncalib_descriptor_modelsearches for the best matches of an uncalibrated descriptor model. It returns a 2D projective transformation matrix (homography) and the score describing the quality of the match.
Except for the camera parameters (CamParam), the uncalibrated and the calibrated case need the same parameters to be adjusted.
Note that the camera parameters, assuming the same setup for the creation of the model and the search, remain the same that were already specified.
Adjusting the detector for the searchπ
Affected parameters
DetectorParamName, DetectorParamValue
The parameters that control the detection, that is, the extraction of the interest points from the image, are adjusted via DetectorParamName and DetectorParamValue.
They correspond to the detector parameters that were already specified during the creation of the model.
In most cases, they should not be changed for the search.
That is, you simply pass an empty tuple ([]) to the parameters.
In rare cases, especially when there are significant illumination changes between the reference image and the search image, you may change the parameter values.
For example, if a search image is extremely dark and 'lepetit' was selected as detector, you can set 'min_score' to a smaller value.
Generally, to test if it is necessary to change any of the parameter values, you can apply the point operator that corresponds to the detector not only to the reference image, which was proposed for the creation of the model, but also to the search image.
Again, about 50 to 450 uniformly distributed points should be extracted to get a good matching result. If the point operator needs different parameters for the reference and the search image, it might be necessary to change the values of the corresponding detector parameters accordingly.
Adjusting the descriptor for the searchπ
Affected parameters
DescriptorParamName, DescriptorParamValue
The parameters that control the determination of the correspondences between the interest points of the search image and those of the model are adjusted via DescriptorParamName and DescriptorParamValue.
Two generic parameters can be set:
- The parameter
'min_score_descr'can be set to a value larger than0.0(but preferably below0.1) to increase the minimal classification score that determines if the individual points are considered as potential matches. Thus, the number of points for further calculations is reduced, so that the speed of the matching can be enhanced. But note that this speed-up is obtained at the cost of a reduced robustness, especially if the number of extracted points is low. - The parameter
'guided_matching'can be switched off by setting it from'on'to'off'. If the guided matching is switched on, which is the default, the robustness of the estimation of the modelβs location is enhanced. In particular, points are extracted from the search image and classified by the descriptor. The points that were accepted by the classification are used to calculate an initial projective 2D (uncalibrated case) or homogeneous 3D (calibrated case) transformation matrix.
This is then used to project all model points into the search image. If a projected model point is near to one of the originally extracted points, that is, independently from its classification, this point is used for the final calculation of the homography that is returned by the matching as 2D projective transformation matrix or 3D pose, respectively.
As typically without the classification more points can be used for the calculations, the obtained homography is more robust.
On the other hand, in some cases the runtime of the matching may increase up to 10%.
Thus, if robustness is less important than speed, 'guided_matching' can be switched off.
The example program %HALCONEXAMPLES%\hdevelop\Applications\Object-Recognition-2D\detect_brochure_pages.hdev is an example for setting 'min_score_descr' to a higher value (0.003) to speed up the matching.
As the number of points in the images is large enough, the matching is still sufficiently robust.
find_uncalib_descriptor_model (ImageGray, ModelIDs[Index2], 'threshold', 600, ['min_score_descr', 'guided_matching'], [0.003, 'on'], 0.25, 1, 'num_points', HomMat2D, Score)
Specifying the similarity of the objectπ
Affected parameter
MinScore
The parameter MinScore specifies the minimum score a potential match must have to be returned as match.
The score is a value for the quality of a match, that is, for the correspondence, or βsimilarityβ, between the model and the search image.
Note that for the descriptor-based matching, different types of score are available for the output parameter Score.
But for the input parameter MinScore, always the score of type 'inlier_ratio' is used.
It calculates the ratio of the number of point correspondences to the number of model points.
In most cases, MinScore should be set to a value of at least 0.1.
To speed up the search, it should be chosen as large as possible, but of course still as small as necessary for the success of the search, as, for example, a value of 1.0 is rather unlikely to be reached by a matching.
Searching for multiple instances of the objectπ
Affected parameter
NumMatches
All you have to do to search for more than one instance of the object is to set the parameter NumMatches to the maximum number of instances you want to find.
If you select the value 0, all matches are returned.
Locating more than one instance, the operators find_uncalib_descriptor_model and find_calib_descriptor_model return the results for the individual model instances concatenated in the output tuples.
That is, the Score, which is typically a single value for a single model instance (except few cases), is now returned as a tuple for which the number of elements corresponds to the number of found model instances.
The number of elements for the projective transformation matrix (HomMat2D) or the 3D pose (Pose), which are tuples already for a single model instance, is the number of elements of the single instance multiplied by the number of found instances.
Selecting a suitable score typeπ
Affected parameter
ScoreType
The parameter ScoreType is used to select the type of score that will be returned in the parameter Score.
Available types are 'num_points' and 'inlier_ratio':
- For
'num_points', the number of point correspondences per instance is returned. As any four correspondences define a mathematically correct homography between two images, this number should be at least10to assume a reliable matching result. - For
'inlier_ratio'the ratio of the number of point correspondences to the number of model points is returned. Although this parameter may have a value between0.0and1.0, a ratio of1.0is rather unlikely to be reached by a matching. Yet, objects having an inlier ratio of less than0.1should be disregarded.
Typically, one of both score types is selected, but it is also possible to pass both types in a tuple. Then, the result for a single found model instance is returned in a tuple as well.
Using the specific results of descriptor-based matchingπ
The descriptor-based matching returns for the uncalibrated case a 2D projective transformation matrix (HomMat2D), for the calibrated case a 3D pose (Pose), and for both cases a score (Score) that evaluates the quality of the returned object location.
The 2D projective transformation matrix and the 3D pose can be used, for example, to transform a structure of the reference image into the search image for a 2D projective transformation matrix or for a 3D pose.
Dealing with multiple matchesπ
If multiple instances of the object are searched and found, the results are concatenated in tuples.
In particular, Score, which is typically a single value for a single model instance is now a tuple for which the number of elements corresponds to the number of found model instances.
The access of the value for a specific instance is similar to the corresponding proceeding for correlation-based matching.
The parameters HomMat2D or Pose are already returned as tuples for a single model instance.
For multiple instances, the values are simply concatenated.
To access them for a specific match, you must know the number of elements returned for a single model instance, which is nine for a single projective transformation matrix and seven for a single 3D pose.
Then, for example, in the tuple returned for the 3D poses, the first seven values belong to the first instance, the next seven values belong to the second instance, etc.
An example for the access of the values for each instance is described in more detail for calibrated perspective deformable matching.