Skip to content

WorkflowπŸ”—

Selecting the model ROIπŸ”—

As a first step of the descriptor-based matching, the region of interest that specifies the template image must be selected.

This region can have an arbitrary shape, that is, it can also contain holes or consist of several parts that are not connected. Thus, it is possible to create an ROI that contains only β€œrobust” parts of the object.

The ROI must be selected so that potentially important interest points are not directly at the border of the region, as the immediate surroundings (neighborhood) of the interest points are needed to obtain the model.

Additionally, the part of the object that is visible in the region of interest needs to be planar. After having selected a suitable ROI, the reduced image is used as template image for the creation of the model.

Creating a suitable perspective deformable modelπŸ”—

Having derived the template image from the reference image, the descriptor model can be created using

Except for the camera parameters and the reference pose, which are needed only for the calibrated case, the parameters that have to be adjusted are the same for both operators. Use the parameters to:

Note that after the creation of the model, the model can still be modified.

Selecting and adjusting the detectorπŸ”—

The interest points that build the model are extracted from the image by the so-called detector. The type of detector is selected via the parameter DetectorType. Available types are 'lepetit', 'harris', and 'harris_binomial', which correspond to the HALCON point operators points_lepetit, points_harris, and points_harris_binomial.

'lepetit' can be used for a very fast extraction of significant points, but the obtained points are not as robust as those obtained with 'harris'. Especially if a template or search image is very dark or has got a low contrast, 'lepetit' is not recommended. 'harris_binomial' is a good compromise between 'lepetit' and 'harris', because it is faster than 'harris' and more robust than 'lepetit'.

For each detector type a set of generic parameters is available that can be adjusted with the parameters DetectorParamName and DetectorParamValue. The first one is used to specify the names of the generic parameters and the second one is used to specify the corresponding values. We recommend to apply tests with the selected point operator before creating the model. That is, you apply the corresponding point operator to the template image and visualize the returned points using, for example, gen_cross_contour_xld.

For a good result, about 50 to 450 points should be uniformly distributed within the template image. If you have found the appropriate parameter setting for the selected point operator, you can set these parameters also for the model in create_calib_descriptor_model or create_uncalib_descriptor_model, respectively.

Note that in most cases the default values of the detectors (DetectorParamName and DetectorParamValue set to []) are sufficient.

Adjusting the descriptorπŸ”—

The currently implemented descriptor uses randomized ferns to classify the extracted points, that is, to build characteristic descriptions of the location and the local gray value neighborhood for the interest points.

The descriptor can be adjusted with the parameters DescriptorParamName and DescriptorParamValue. The first one is used to specify the names of the generic parameters that have to be adjusted and the second one is used to specify the corresponding values.

The parameters can be divided into parameters that control the size of the descriptor and thus allow to control the detection robustness, speed, and memory consumption, and in parameters that control the simulation, especially the spatial range in which the model views are trained. The size of the descriptor is controlled by the following parameters:

  • 'depth' specifies the depth of the classification fern. Interest points can better be discriminated when selecting a higher depth. On the other hand, a higher depth leads to an increasing runtime.
  • 'number_ferns' specifies the number of used fern structures. Using many fern structures leads to a better robustness but also to an increasing runtime.
  • 'patch_size' specifies the side length of the quadratic neighborhood that is used to describe the individual interest point. Again, a too large value can disadvantageously influence the runtime.

The selection of values for the depth and the number of ferns depends on your specific requirements. If a fast online matching is required, few ferns with a large depth are recommended. If a robust matching result is needed, many ferns are needed and a large depth may additionally increase the robustness, although it might also significantly increase the runtime of the matching. If the memory consumption is critical, many ferns with a small depth are recommended. For many applications, a trade-off between the different requirements will be needed.

The simulation, that is, the training of the model is controlled by the following parameters:

  • 'tilt' is used to switch 'on' or 'off' the projective transformations during the simulation phase, which leads either to an enhanced robustness of the model or to a speed-up of the training.
  • 'min_rot' and 'max_rot' define the range for the angle of rotation around the normal vector of the model.
  • 'min_scale' and 'max_scale' define the scale range of the model.

Restricting the angle of rotation and the scale to small ranges can significantly speed up the training. Note that the model can only be found during the later applied matching if its angle and scale are within the trained scope. Further, the training is faster for small images. Thus, you can speed up the training by selecting a small reference image and small template images and setting 'tilt' to 'off'. Note also that the reference image and the search image should have the same size.

An example for the restriction of the orientation and scale of the model is given in the HDevelop example program %HALCONEXAMPLES%\hdevelop\Applications\Object-Recognition-2D\detect_brochure_pages.hdev, which creates an uncalibrated descriptor model for the matching of different brochure pages. There, the orientation is restricted to an angle range of Β± 90Β° ('default' is Β± 180Β°) and the scale range is changed to a scaling factor between 0.2 and 1.1 ('default' is between 0.5 and 1.4).

create_uncalib_descriptor_model (ImageReduced, 'harris_binomial', [], [], ['min_rot', 'max_rot', 'min_scale', 'max_scale'], [-90, 90, 0.2, 1.1], 42, ModelID)

Specifying the camera parameters and the reference poseπŸ”—

For the calibrated matching, the internal camera parameters (CamParam) and a reference pose (ReferencePose) have to be specified. We recommend to obtain both using a camera calibration. Other approaches for determining the pose of the object plane comprise a manual measurement of the extents of the model, which is rather intricate and often inaccurate, stereo vision (see Stereo system), or 3D laser triangulation, for example, using sheet of light (see Sheet of light).

Inspecting and modifying the descriptor modelπŸ”—

If you want to visually inspect an already created model, you can get the coordinates of the interest points that are contained in it using get_descriptor_model_points with Set set to 'model'.

get_descriptor_model_points (ModelID, 'model', 'all', Row_D, Col_D)

To inspect the current parameter values of a model, you query them with get_descriptor_model_params. This may be necessary if during the creation of the model an automatic parameter selection was used or if the model was created within another program, saved to file with write_descriptor_model, and read from this file in the current program with read_descriptor_model. Additionally, you can query the coordinates of the origin of the model using get_descriptor_model_origin.

After the creation of the model and before you search the object in a search image, you can further modify the model. In particular, you can apply set_descriptor_model_origin to change its origin. But note that this is not recommended because the accuracy of the matching result may decrease, which is shown in more detail for shape-based matching.

If you need to modify the point of reference and you want to apply a calibrated matching, see corresponding description for perspective deformable matching. There, the relation between the reference pose, the model pose, and an offset that is manually applied to the point of reference is introduced.

Note that after the matching, you can use get_descriptor_model_points also to query the interest points of a specific match. In this case, Set must be set to 'search' instead of 'model'. Here, the interest points of the first found model instance (index 0) are queried.

get_descriptor_model_points (ModelIDs[Index2], 'search', 0, Row, Col)

Additionally, after the matching, get_descriptor_model_results can be used to query selected numerical results that were accumulated during the search like the scores for the correspondences between the individual search points and model points (ResultNames set to 'point_classification').