Skip to content

Position recognition🔗

2D vs. 3D matching approaches🔗

The different matching approaches are suitable for different applications. Some are suitable only for 2D objects imaged from an orthogonal view, others can also handle perspective distortions, and some approaches even match 3D shapes in a full 3D space.

Note that with the term “2D object” a fixed view on a planar object part is meant and not the actual tangible object, which in reality is naturally three-dimensional. In contrast, the “3D object” is an object that is viewed from arbitrary directions.

  • 2D objects, imaged from an orthogonal view:
    The correlation-based, shape-based, and local deformable matching can be used to find 2D objects in images.
    The objects must be taken from an orthogonal view for the reference image as well as for the search images.
    Theoretically, you can also use the perspective deformable or the descriptor-based matching to find orthogonally imaged 2D objects, but as these are not as fast as the strict 2D approaches, they are recommended rather for the case that the objects must be imaged from a perspective view.

  • 2D objects, imaged from a perspective view:
    The perspective deformable and the descriptor-based matching can be used to find planar object parts, too.
    But in addition to the approaches used for the orthogonal view, the objects may be perspectively deformed.
    Additionally, if a camera calibration was applied, not only the 2D position and orientation but the 3D pose of the object can be derived.
    Although both approaches can be used also for orthogonally imaged 2D objects, they are recommended rather for the perspective case, as they are not as fast as the strict 2D approaches.

  • 3D objects:
    The 3D matching searches for “real” 3D objects in 2D images.
    Here, different approaches are available.
    For example, for the shape-based 3D matching, no reference image is used to create the model but a synthetic 3D model, in particular a DXF CAD model, must be provided.
    If a 3D object contains a characteristic planar part which is visible in all search images, alternatively the calibrated perspective 2D approaches, that is, the perspective deformable or the descriptor-based matching, can be used, which are more convenient and significantly faster.

The 3D matching is needed only if more than one planar part of the object is needed to differentiate the object from other image parts.

Model described by Orthogonal view Arbitrary view (or deformations)
gray-values correlation−based
shape contours shape−based
local deformable
perspective deformable
perspective deformable
set of points descriptor−based matching
Model described by Orthogonal view Arbitrary view (or deformations)
gray-values
shape contours perspective deformable perspective deformable
3D matching
set of points descriptor−based descriptor−based

2D vs. 3D objects🔗

If you search for a complex 3D object that differs from other objects in all three dimensions (see, for example, ), the selection of the appropriate matching approach is easy as you have to use the 3D matching.

If the 3D object contains a unique but planar part that significantly differs from the other structures in the expected images or if you search for a planar object that may be oriented arbitrarily in the 3D space you can also use the perspective deformable or the descriptor-based matching, which can handle perspective distortions of 2D objects. Both approaches are faster and more convenient to use than the 3D matching.

Which of both perspective approaches to select depends on the specific application. If the object of interest is expected to be anisotropically scaled in the search images, only the perspective deformable matching is suitable. If it is expected to be only translated and rotated, both approaches are suitable and the appearance of the object in the images must be taken into account. The most important difference between both approaches is the way the object is modeled. The perspective deformable matching describes the model by the shapes of the object’s contours. Thus, it is suitable for objects that contain clearly visible contours (see, for example, ).

In contrast, the descriptor-based matching describes the model by interest points. Thus, for objects that are rather characterized by an arbitrary but fixed texture (see, for example, ), it is most probably to be preferred.