Skip to content

2D position recognitionπŸ”—

What is matching?πŸ”—

With matching, HALCON provides a method for the robust location of objects in images, which can be used for many different applications. To suit the different requirements of the applications, different approaches are available.

All approaches consist of a small set of operators for which only a few parameters have to be adjusted. Further, none of the approaches requires an explicit segmentation of the objects in the images. Thus, you can successfully locate your objects even if you have no special knowledge in machine vision.

The main idea of matching is to use a prototype object (template), create a model of it and search the model in other images. For most matching tasks you obtain the model from a reference image that shows the object of interest. To suppress other structures or objects that are contained in this image, the image is reduced to a region of interest (ROI) that only contains the object and which may have an arbitrary shape. The reduced image is the template image from which the model is created by an approach-specific operator. Another approach-specific operator uses the obtained model to find the object in different search images. That is, it searches for image structures that match the model (within small tolerances).

The different matching approaches provided by HALCON differ amongst others in the image structures that are used to build the model. Some matching approaches use, for example, the relations of gray values to their surrounding (neighborhood) to build a model. Others use, for example, the shapes of contours (see Table 1). The result of the matching is in each case information about the position, for most approaches the orientation, and for some approaches also the scale of the found object in the search image.

Table 1: Matching using shapes of contours.
Image of an attention sign A (shape) model is derived from the image in different resolutions (pyramid levels). Instance of the model is located in an image.

Quick guide to the 2D matching approachesπŸ”—

Transformations that can be handled by the matching approachesπŸ”—

Geometric transformations of an object
Translation Rotation Scaling (uniform) Scaling (anisotropic)
Approach Translation (2D) Rotation (2D) Scaling (uniform) Scaling (anisotropic)
correlation-based βœ… βœ…
shape-based βœ… βœ… βœ… βœ…
local deformable βœ… βœ… βœ… βœ…
perspective deformable βœ… βœ… βœ… βœ…
descriptor-based βœ… βœ… βœ…

Transformation parameters returned by the matching approachesπŸ”—

Approach Position (2D) Angle (2D) Scale (uniform) Scale (anisotropic) Projective transformation matrix Pose (3D)
correlation-based βœ… βœ…
shape-based βœ… βœ… βœ… βœ…
local deformable βœ…
perspective deformable βœ… βœ…
descriptor-based βœ… βœ…

Characteristics of the appearance the matching approaches can cope withπŸ”—

Examples for different appearances
Occlusion (missing contours) Clutter (additional contours) Illumination changes
Approach Occlusion Clutter Illumination change Texture Color Defocus
correlation-based βœ…
(only linear)
βœ… βœ…
shape-based βœ… βœ… βœ… βœ… βœ…
local deformable βœ… βœ… βœ… βœ…
perspective deformable βœ… βœ… βœ… βœ…
descriptor-based βœ… βœ… βœ… βœ…
(needed)

First decisions for 2D objects in orthogonal viewπŸ”—

If you can image planar objects from an orthogonal view and only the 2D transformation parameters of the found object instances are needed, the perspective deformable or the descriptor-based matching would lead to the desired result as well, but the approaches that are strictly restricted to two dimensions are to be preferred as they are significantly faster. The 2D approaches comprise the correlation-based, shape-based, and local deformable matching.

Three of theses approaches are used only in specific situations:

  • The local deformable matching is applied only if the object of interest is locally deformed, for example, because of a contorted surface (see Figure 1).
    In contrast to the other 2D approaches, it returns only the position but no orientation for the found model instance.
    In exchange, it allows to rectify the image part containing the deformed object and returns a vector field that describes the actual deformations.

In most orthogonal 2D cases, you have to select one of the two remaining approaches, that is, either the shape-based matching or the correlation-based matching.

Shape-based vs. correlation-based matchingπŸ”—

To decide if the shape-based or the correlation-based matching is more appropriate for your specific application, you should further investigate the requirements of your application.

For example, you should know which transformation parameters are needed to describe the relation of the object in the search image to the object described by the model. Will the object be only translated and rotated or also scaled? If a scaling is needed, correlation-based matching can not be used, but one of the shape-based matching approaches must be used. Here, you can further choose between approaches that use uniform scaling or anisotropic scaling, that is, a scaling with different scaling factors for the x- and y-direction.

Additionally, the appearance of the object may change from image to image. Reasons can be, for example, occlusions, clutter, or illumination changes that may lead to a changing polarity of the object. Furthermore, the images may be defocused or the object is a complex pattern in front of a complex background, that is, it is textured. To get a robust and fast result, the changes of the object’s appearance should be minimized as much as possible already when imaging the object. Nevertheless, sometimes distortions like occlusions, clutter, or defocus can not be avoided. These distortions have to be involved into the decision which approach to use for a specific application.

In particular, shape-based matching should be chosen if occlusions (see, for example, Figure 2) or clutter can not be avoided or if a matching of objects with changing color is applied.

Figure 2: Occlusions can be handled by shape-based matching.

In contrast, correlation-based matching is suitable for objects with a random and changing texture or for objects with a slightly changing shape (see Figure 3). Additionally, correlation-based matching is to be preferred when handling strongly defocused images (see Figure 4).

Figure 3: Changing shapes can be handled by correlation-based matching.

Figure 4: Defocused shapes can be handled by correlation-based matching.

Programming examplesπŸ”—

%HALCONEXAMPLES%/hdevelop/Matching/Correlation-Based/ncc_matching_workflow.hdev

Check whether the logo (yellow) is visible in the image. Found match (green).

%HALCONEXAMPLES%/hdevelop/Matching/Shape-Based/alignment_for_ocr_in_semiconductor.hdev

Model (yellow) used for alignment to specify search region (green). The found text (colored) and the recognized characters (yellow).

%HALCONEXAMPLES%/hdevelop/Matching/Deformable/find_local_deformable_model.hdev

β€˜MVTec’ logo used for the model creation. Deformed logo instance overlaid by the model contours.

%HALCONEXAMPLES%/hdevelop/Applications/Traffic-Monitoring/detect_road_signs.hdev

Figure 5: A dead end road sign is located via uncalibrated perspective deformable matching.

%HALCONEXAMPLES%/hdevelop/Applications/Object-Recognition-2D/detect_brochure_pages.hdev

Figure 6: Different pages of a HALCON brochure are located via descriptor-based matching.