Skip to content

OptimizationπŸ”—

Restricting the search to a region of interestπŸ”—

The obvious way to restrict the search space and thus speed up the matching is to apply the operator find_local_deformable_model not to the whole image but only to an ROI. The corresponding procedure is explained in more detail for shape-based matching. For local deformable matching you simply have to use find_local_deformable_model.

Restricting the range of orientation and scaleπŸ”—

Affected parameters

AngelExtent, AngleStart, ScaleCMax, ScaleCMin, ScaleRMax, ScaleRMin

When calling find_local_deformable_model, you can limit the orientation and scale ranges with the parameters AngleStart, AngleExtent, ScaleMin, and ScaleMax.

This is useful if you can restrict these ranges by other information, which can, for example, be obtained by suitable image processing operations.

Specifying the visibility of the objectπŸ”—

Affected parameter

MinScore

With the parameter MinScore you can specify how much of the model must be visible. A typical use of this mechanism is to allow a certain degree of occlusion. The parameter is similar to the corresponding parameter for shape-based matching.

Trading thoroughness vs. speedπŸ”—

Affected parameter

Greediness

With the parameter Greediness you can influence the search algorithm itself and thereby trade thoroughness against speed.

Searching for multiple instances of the ObjectπŸ”—

Affected parameters

MaxOverlap, NumMatches

All you have to do to search for more than one instance of the object is to set the parameter NumMatches to the maximum number of instances you want to find. If you select the value 0, all matches are returned.

Locating more than one instance, the operator find_local_deformable_model returns the results for the individual model instances concatenated in the output tuples.

Note that a search for multiple objects is only slightly slower than a search for a single object.

A second parameter, MaxOverlap, lets you specify how much two matches may overlap (as a fraction). This parameter is similar to the corresponding parameter of shape-based matching.

Restricting the number of pyramid levelsπŸ”—

Affected parameter

NumLevels

The parameter NumLevels, which you already specified when creating the model, allows you to use a different (in most cases a more restrictive) value in the search process.

By using the value 0 for NumLevels, the value specified when creating the model is used.

Specifying the iconic objects returned by the matchingπŸ”—

Affected parameter

ResultType

By default, a position and a score are returned for each match.

If you additionally want to obtain some iconic results, you can set the parameter ResultType to 'image_rectified', 'vector_field', and 'deformed_contours'. With 'image_rectified' you get a rectified version of the part of the search image that corresponds to the bounding box of the ROI that was used to create the model (RectifiedImage), with 'vector_field' you get the vector field that describes the deformations of the matched model instance (VectorField), and with 'deformed_contours' you get the contours of the deformed model instance (DeformedContours).

The individual result types are introduced here in more detail.

Adjusting generic parametersπŸ”—

Affected parameters

GenParamName, GenParamValue

Besides the parameters that correspond more or less to those used for shape-based matching, local deformable matching allows to adjust some generic parameters that are set via GenParamName and GenParamValue.

Typically, there is no need to adjust them. But if, for example, a low accuracy of the matching is sufficient, you can speed up the matching by a reduction or deactivation of the subpixel precision or by changing the discretization steps for the orientation or scale that were set during the creation of the model.

Additionally, you can adjust the expected β€œsmoothness” of the deformation. In particular, if the deformations vary locally, that is, if the deformations are expected to be rather different within a small neighborhood, the value of 'deformation_smoothness' may be set to a smaller value, whereas for an object with an expected β€œsmooth” transition between the deformations the parameter may be set to a higher value.

Furthermore, you can expand the border of the optionally returned rectified image. By default, the section of the search image that is rectified corresponds to the bounding box of the ROI that was used to create the model.

If a larger section is needed, for example, if neighboring parts that are visible in the search image are needed as well for a further processing of the image, you can expand the border of the rectified image with the parameters 'expand_border', 'expand_border_top', 'expand_border_bottom', 'expand_border_left', or 'expand_border_right', respectively. If, for example, in the example program only the letters β€œMVT” would have been contained in the model, but the letters β€œec” would have been needed as well, the parameter 'expand_border_right' could have been set to 50 to expand the image section by 50 pixels.

Note

The rectification is reliable only for the image section that corresponds to the model ROI, as for the neighboring parts of the search image an extrapolation is applied.

Thus, the larger the expanded border is, the less accurate is the result towards the image border.

For further information, see find_local_deformable_model.

Using the specific results of local deformable matchingπŸ”—

The results of local deformable matching differ from those of the other matching approaches. Though it returns also a position and a score, it does not return an orientation or even a scale. Instead, a set of iconic objects can be returned that helps to inspect the deformations of the found object instance.

Depending on the selected values of ResultType, the following iconic objects are returned:

Similar to shape-based matching, local deformable matching can return multiple instances of a model.

About rectified imagesπŸ”—

The first of the iconic objects that can be returned by the local deformable matching is the rectified part of the search image that corresponds to the bounding box of the ROI that was used to create the model (see Figure 1).

Figure 1: Search image (left) and rectified image part (right).

It can be used, for example, as was introduced in the first example to show the difference between the rectified image and the corresponding part of the reference image. Another use may be that the rectified image is needed for a further image processing task.

Note

Note that if a larger section of the search image is needed for the further processing, it is possible to expand the borders of ImageRectified.

About vector fieldsπŸ”—

Another iconic object that is returned by the matching is a vector field that can be used, for example, to visualize the deformations. For that, VectorField can be transformed into a grid as was shown in the first example.

Note that to directly display the vector field is not reasonable, as in contrast to a vector field that is returned, for example, by optical flow, no relative but absolute coordinates are returned. In particular, the vector field that is returned, for example, by optical_flow_mg describes the relative movement of the points from one image to another image of the same size. The components of the vector field that can be queried with vector_field_to_real contain for each point the relative movement in row and column direction.

In contrast, the vector field that is returned by local deformable matching describes the movement of points from the model, or more precisely from the bounding box that surrounds the template image, to a search image that typically is larger than the model. Thus, instead of the relative movement of each point, the vector field contains for each model point the corresponding absolute coordinates of the search image.

Besides visualization purposes, the vector field can be used to get a rectification map that can be used for other applications as well. If, for example, the deformations of a model instance do not stem from a deformed object but from significant camera distortions, you may use the returned vector field to compensate for the camera distortions in other images.

For that, the model either must be a synthetic model or it must be created from an image that was made with a camera without significant distortions. The vector field that is returned by the matching in a search image that is made by the distorted camera can then be transformed into a map with convert_map_type. This map can then be used to rectify other images that were made with the same camera.

About deformed contoursπŸ”—

At last, the deformed contours can be returned.

Similar to the model contours the DeformedContours can be used for visualization purposes (seeFigure 2) or for a visual inspection of the difference between the model and the found model instance.

For the latter, you can either overlay the matching result with the model contours or overlay the model with the deformed contours. Whereas the model contours by default are located at the origin of the image and thus must be transformed for a proper visualization the deformed contours are located already at the position of the match.

Figure 2: Search image (left) and rectified image part (right).

Dealing with multiple matchesπŸ”—

If multiple instances of the object are searched and found, the results are concatenated in tuples. For example, Score, which is a single value for a single model instance, is now a tuple for which the number of elements corresponds to the number of found model instances. The access of the value for a specific instance is similar to the proceeding described for correlation-based matching.

The iconic output object DeformedContours is already returned as a tuple for a single model instance. For multiple instances, the values are simply concatenated. To access them for a specific match, you must know the number of elements returned for a single model instance. This can be obtained by calling count_obj for the model contours that can be queried with get_deformable_model_contours after the model creation. If, for example, the model consists of five contours, in the tuple returned for the deformed contours, the first five values belong to the first instance, the next five values belong to the second instance, etc. An example for the access of the values for each instance is described in more detail for calibrated perspective deformable matching .