Operator Reference
create_shape_model_3d (Operator)
create_shape_model_3d — Prepare a 3D object model for matching.
Signature
create_shape_model_3d( : : ObjectModel3D, CamParam, RefRotX, RefRotY, RefRotZ, OrderOfRotation, LongitudeMin, LongitudeMax, LatitudeMin, LatitudeMax, CamRollMin, CamRollMax, DistMin, DistMax, MinContrast, GenParamName, GenParamValue : ShapeModel3DID)
Description
The operator create_shape_model_3d prepares a 3D object
model, which is passed in ObjectModel3D, as a 3D shape
model used for matching. The 3D object model must previously been
read from a file by using read_object_model_3d.
The 3D shape model is generated by computing different views of the
3D object model within a user-specified pose range. The views are
automatically generated by placing virtual cameras around the 3D
object model and projecting the 3D object model into the image plane
of each virtual camera position. For each such obtained view a 2D
shape representation is computed. Thus, for the generation of the 3D shape
model, no images of the object are used but only the 3D object model,
which is passed in ObjectModel3D. The shape representations
of all views are stored in the 3D shape model, which is returned in
ShapeModel3DID. During the matching process with
find_shape_model_3d, the shape representations are used to
find out the best-matching view, from which the pose is subsequently
refined and returned.
In order to create the model views correctly, the camera parameters
of the camera that will be used for the matching must be passed in
CamParam. The camera parameters are necessary, for example,
to determine the scale of the projections by using the actual focal
length of the camera. Furthermore, they are used to treat radial
distortions of the lens correctly. Consequently, it is essential
to calibrate the camera by using calibrate_cameras before
creating the 3D shape model. On the one hand, this is necessary to
obtain accurate poses from find_shape_model_3d. On the other
hand, this makes the 3D matching applicable even when using lenses
with significant radial distortions.
The pose range within which the model views are generated can be
specified by the parameters RefRotX, RefRotY,
RefRotZ, OrderOfRotation, LongitudeMin,
LongitudeMax, LatitudeMin, LatitudeMax,
CamRollMin, CamRollMax, DistMin, and
DistMax.
Note that the model will only be recognized during
the matching if it appears within the specified pose range.
The parameters are described in the following:
Before computing the views, the origin of the coordinate system of
the 3D object model is moved to the reference point of the 3D object
model, which is the center of the smallest enclosing axis-parallel
cuboid and can be queried by using
get_object_model_3d_params. The virtual cameras, which are
used to create the views, are arranged around the 3D object model in
such a way that they all look at the origin of the coordinate
system, i.e., the z axes of the cameras pass through the origin. The
pose range can then be specified by restricting the views to a
certain quadrilateral on the sphere around the origin. This
naturally leads to the use of the spherical coordinates longitude,
latitude, and radius. The definition of the spherical coordinate
system is chosen such that the equatorial plane corresponds to the
xz plane of the Cartesian coordinate system with the y axis pointing
to the south pole (negative latitude) and the negative z axis
pointing in the direction of the zero meridian (see
convert_point_3d_spher_to_cart or
convert_point_3d_cart_to_spher for further details about the
conversion between Cartesian and spherical coordinates). The
advantage of this definition is that a camera with the pose
[0,0,z,0,0,0,0] has its optical center at longitude=0, latitude=0,
and radius=z. In this case, the radius represents the distance of
the optical center of the camera to the reference point of the 3D
object model.
The longitude range, for which views are to be generated, can be
specified by LongitudeMin and LongitudeMax, both
given in radians. Accordingly, the latitude range can be
specified by LatitudeMin and LatitudeMax, also given
in radians. LongitudeMin and LongitudeMax
are adjusted to maintain a range of 360° .
If an adjustment is possible, LongitudeMin and the range are
preserved. The minimum and maximum distance between the camera center
and the model reference point is specified by DistMin and
DistMax.
Thereby, the model origin is in the center of the smallest enclosing cuboid
and does not necessarily coincide with the origin of the CAD coordinate
system.
Note that the unit of the distance must be meters
(assuming that the parameter Scale has been correctly set
when reading the CAD file with read_object_model_3d).
Finally, the minimum and the maximum camera roll angle can be
specified in CamRollMin and CamRollMax. This
interval specifies the allowable camera rotation around its z axis
with respect to the 3D object model. If the image plane is parallel
to the plane on which the objects reside and if it is known that the
object may rotate in this plane only in a restricted range, then it
is reasonable to specify this range in CamRollMin and
CamRollMax. In all other cases the interpretation of the
camera roll angle is difficult, and hence, it is recommended to set
this interval to . Note that the larger
the specified pose range is chosen the more memory the model will
consume (except from the range of the camera roll angle) and the
slower the matching will be.
The orientation of the coordinate system of the 3D object model is
defined by the coordinates within the CAD file that was read by
using read_object_model_3d. Therefore, it is reasonable
to previously rotate the 3D object model into a reference
orientation such that the view that corresponds to longitude=0 and
latitude=0 is approximately at the center of the pose range. This
can be achieved by passing appropriate values for the reference
orientation in RefRotX, RefRotY, RefRotZ,
and OrderOfRotation. The rotation is performed around the
axes of the 3D object model, which origin was set to the reference
point. The longitude and latitude range can then be interpreted as a
variation of the 3D object model pose around the reference
orientation. There are two possible ways to specify the reference
orientation. The first possibility is to specify three rotation
angles in RefRotX, RefRotY, and RefRotZ
and the order in which the three rotations are to be applied in
OrderOfRotation, which can either be 'gba' or
'abg'. The second possibility is to specify the three
components of the Rodriguez rotation vector in RefRotX,
RefRotY, and RefRotZ. In this case,
OrderOfRotation must be set to 'rodriguez' (see
create_pose for detailed information about the order of the
rotations and the definition of the Rodriguez vector).
Thus, two transformations are applied to the 3D object model before computing the model views within the pose range. The first transformation is the translation of the origin of the coordinate systems to the reference point. The second transformation is the rotation of the 3D object model to the desired reference orientation around the axes of the reference coordinate system. By combining both transformations one obtains the reference pose of the 3D shape model. The reference pose of the 3D shape model thus describes the pose of the reference coordinate system with respect to the coordinate system of the 3D object model defined by the CAD file. Let t = (x,y,z)' be the coordinates of the reference point of the 3D object model and R be the rotation matrix containing the reference orientation. Then, a point given in the 3D object model coordinate system can be transformed to a point in the reference coordinate system of the 3D shape model by applying the following formula:
This transformation can be expressed by a homogeneous 3D
transformation matrix or alternatively in terms of a 3D pose. The
latter can be queried by passing 'reference_pose' for the
parameter GenParamName of the operator
get_shape_model_3d_params. The above formula can be best
imagined as a pose of pose type 8, 10, or 12, depending on the value
that was chosen for OrderOfRotation (see
create_pose for detailed information about the different
pose types). Note, however, that get_shape_model_3d_params
always returns the pose using the pose type 0. Finally, poses that
are given in one of the two coordinate systems can be transformed to
the other coordinate system by using
trans_pose_shape_model_3d.
Furthermore, it should be noted that the reference coordinate system
is introduced only to specify the pose range in a convenient
way. The pose resulting from the 3D matching that is performed with
find_shape_model_3d always refers to the original 3D object
model coordinate system used in the CAD file.
With MinContrast, it can be determined which edge contrast
the model must at least have in the recognition performed by
find_shape_model_3d. In other words, this parameter
separates the model from the noise in the image. Therefore, a good
choice is the range of gray value changes caused by the noise in the
image. If, for example, the gray values fluctuate within a range of
10 gray levels, MinContrast should be set to 10. If
multichannel images are used for the search images, the noise in one
channel must be multiplied by the square root of the number of
channels to determine MinContrast. If, for example, the
gray values fluctuate within a range of 10 gray levels in a single
channel and the image is a three-channel image, MinContrast
should be set to 17. If the model should be recognized in very low
contrast images, MinContrast must be set to a
correspondingly small value. If the model should be recognized even
if it is severely occluded, MinContrast should be slightly
larger than the range of gray value fluctuations created by noise in
order to ensure that the pose of the model is extracted robustly and
accurately by find_shape_model_3d.
The parameters described above are application-dependent and must be
always specified when creating a 3D shape model. In addition, there
are some generic parameters that can optionally be used to influence
the model creation. For most applications these parameters need not
to be specified but can be left at their default values. If desired,
these parameters and their corresponding values can be specified by
using GenParamName and GenParamValue,
respectively. The following values for GenParamName are
possible:
- 'num_levels':
-
For efficiency reasons the model views are generated on multiple pyramid levels. On higher levels fewer views are generated than on lower levels. With the parameter 'num_levels' the number of pyramid levels on which model views are generated can be specified. It should be chosen as large as possible because by this the time necessary to find the model is significantly reduced. On the other hand, the number of levels must be chosen such that the shape representations of the views on the highest pyramid level are still recognizable and contain a sufficient number of points (at least four). If not enough model points are generated for a certain view, the view is deleted from the model and replaced by a view on a lower pyramid level. If for all views on a pyramid level not enough model points are generated, the number of levels is reduced internally until for at least one view enough model points are found on the highest pyramid level. If this procedure would lead to a model with no pyramid levels, i.e., if the number of model points is too small for all views already on the lowest pyramid level,
create_shape_model_3dreturns an error message. If 'num_levels' is set to 'auto' (default value),create_shape_model_3ddetermines the number of pyramid levels automatically. In this case all model views on all pyramid levels are automatically checked whether their shape representations are still recognizable. If the shape representation of a certain view is found to be not recognizable, the view is deleted from the model and replaced by a view on a lower pyramid level. Note that if 'num_levels' is set to 'auto', the number of pyramid levels can be different for different views. In rare cases, it might happen thatcreate_shape_model_3ddetermines a value for the number of pyramid levels that is too large or too small. If the number of pyramid levels is chosen too large, the model may not be recognized in the image or it may be necessary to select very low parameters forMinScoreorGreedinessinfind_shape_model_3din order to find the model. If the number of pyramid levels is chosen too small, the time required to find the model infind_shape_model_3dmay increase. In these cases, the views on the pyramid levels should be checked by using the output ofget_shape_model_3d_contours.Suggested values: 'auto', 3, 4, 5, 6
Default: 'auto'
- 'fast_pose_refinement':
-
The parameter specifies whether the pose refinement during the search with
find_shape_model_3dis sped up. If 'fast_pose_refinement' is set to 'false', for complex models with a large number of faces the pose refinement step might amount to a significant part of the overall computation time. If 'fast_pose_refinement' is set to 'true', some of the calculations that are necessary during the pose refinement are already performed during the model generation and stored in the model. Consequently, the pose refinement during the search will be faster. Please note, however, that in this case the memory consumption of the model may increase significantly (typically by less than 30 percent). Further note that the resulting poses that are returned byfind_shape_model_3dmight slightly differ depending on the value of 'fast_pose_refinement', because internally the pose refinement is approximated if the parameter is set to 'true'.List of values: 'true', 'false'
Default: 'true'
- 'lowest_model_level':
-
In some cases the model generation process might be very time consuming and the memory consumption of the model might be very high. The reason for this is that in these cases the number of views, which must be computed and stored in the model, is very high. The larger the pose range is chosen and the larger the objects appear in the image (measured in pixels) the more views are necessary. Consequently, especially the use of large images (e.g., images exceeding a size of 640×480 ) can result in very large models. Because the number of views is highest on lower pyramid levels, the parameter 'lowest_model_level' can be used to exclude the lower pyramid levels from the generation of views. The value that is passed for 'lowest_model_level' determines the lowest pyramid level down to which views are generated and stored in the 3d shape model. If, for example, a value of 2 is passed for large models, the time to generate the model as well as the size of the resulting model is reduced to approximately one third of the original values. If 'lowest_model_level' is not passed, views are generated for all pyramid levels, which corresponds to the behavior when passing a value of 1 for 'lowest_model_level'. If for 'lowest_model_level' a value larger than 1 is passed, in
find_shape_model_3dthe tracking of matches through the pyramid will be stopped at this level. However, if infind_shape_model_3da least-squares adjustment is chosen for pose refinement, the matches are refined on the lowest pyramid level using the least-squares adjustment. Note that for different values for 'lowest_model_level' different matches might be found during the search. Furthermore, the score of the matches depends on the chosen method for pose refinement. Also note that the higher 'lowest_model_level' is chosen the higher the portion of the refinement step with respect to the overall run-time offind_shape_model_3dwill be. As a consequence for higher values of 'lowest_model_level' the influence of the generic parameter 'fast_pose_refinement' (see above) on the runtime will increase. A large value for 'lowest_model_level' on the one hand may lead to long computation times offind_shape_model_3dif 'fast_pose_refinement' is switches off ('false'). On the other hand it may lead to a decreased accuracy if 'fast_pose_refinement' is switches on ('true') because in this mode the pose refinement is only approximated. Therefore, the value for 'lowest_model_level' should be chosen as small as possible. Furthermore, 'lowest_model_level' should be chosen small enough such that the edges of the 3D object model are still observable on this level.Suggested values: 1, 2, 3
Default: 1
- 'optimization':
-
For models with particularly large model views, it may be useful to reduce the number of model points by setting 'optimization' to a value different from 'none'. If 'optimization' = 'none', all model points are stored. In all other cases, the number of points is reduced according to the value of 'optimization'. If the number of points is reduced, it may be necessary in
find_shape_model_3dto set the parameterGreedinessto a smaller value, e.g., 0.7 or 0.8. For models with small model views, the reduction of the number of model points does not result in a speed-up of the search because in this case usually significantly more potential instances of the model must be examined. If 'optimization' is set to 'auto',create_shape_model_3dautomatically determines the reduction of the number of model points for each model view.List of values: 'auto', 'none', 'point_reduction_low', 'point_reduction_medium', 'point_reduction_high'
Default: 'auto'
- 'metric':
-
This parameter determines the conditions under which the model is recognized in the image. If 'metric' = 'ignore_part_polarity', the contrast polarity is allowed to change only between different parts of the model, whereas the polarity of model points that are within the same model part must not change. Please note that the term 'ignore_part_polarity' is capable of being misunderstood. It means that polarity changes between neighboring model parts do not influence the score, and hence are ignored. Appropriate model parts are automatically determined. The size of the parts can be controlled by the generic parameter 'part_size', which is described below. Note that this metric only works for one-channel images. Consequently, if the model is created by using this metric and searched in a multi-channel image by using
find_shape_model_3dan error will be returned. If 'metric' = 'ignore_local_polarity', the model is found even if the contrast polarity changes for each individual model point. This metric works for one-channel images as well as for multi-channel images. The metric 'ignore_part_polarity' should be used if the images contain strongly textured backgrounds or clutter objects, which might result in wrong matches. Note that in general the scores of the matches that are returned byfind_shape_model_3dare lower for 'ignore_part_polarity' than for 'ignore_local_polarity'. This should be kept in mind when choosing the right value for the parameterMinScoreoffind_shape_model_3d.List of values: 'ignore_local_polarity', 'ignore_part_polarity'
Default: 'ignore_local_polarity'
- 'part_size':
-
This parameter determines the size of the model parts that is used when 'metric' is set to 'ignore_part_polarity' (see above). The size must be specified in pixels and should be approximately twice as large as the size of the background texture in the image. For example, if an object should be found in front of a chessboard with black and white squares of size 5×5 pixels, 'part_size' should be set to 10. Note that higher values of 'part_size' might also decrease the scores of correct instances especially when searching for objects with shiny or reflective surfaces. Therefore, the risk of missing correct instances might increase if 'part_size' is set to a higher value. If 'metric' is set to 'ignore_local_polarity', the value of 'part_size' is ignored.
Suggested values: 2, 3, 4, 6, 8, 10
Default: 4
- 'min_face_angle':
-
3D edges are only included in the shape representations of the views if the angle between the two 3D faces that are incident with the 3D object model edge is at least 'min_face_angle'. If 'min_face_angle' is set to 0.0, all edges are included. If 'min_face_angle' is set to (equivalent to 180 degrees), only the silhouette of the 3D object model is included. This parameter can be used to suppress edges within curved surfaces, e.g., the surface of a cylinder or cone. Curved surfaces are approximated by multiple planar faces. The edges between such neighboring planar faces should not be included in the shape representation because they also do not appear in real images of the model. Thus, 'min_face_angle' should be set sufficiently high to suppress these edges. The effect of different values for 'min_face_angle' can be inspected by using
project_object_model_3dbefore callingcreate_shape_model_3d. Note that if edges that are not visible in the search image are included in the shape representation, the performance (robustness and speed) of the matching may decrease considerably.Suggested values: 'rad(10)', 'rad(20)', 'rad(30)', 'rad(45)'
Default: 'rad(30)'
- 'min_size':
-
This value determines a threshold for the selection of significant model components based on the size of the components, i.e., connected components that have fewer points than the specified minimum size are suppressed. This threshold for the minimum size is divided by two for each successive pyramid level.
Suggested values: 'auto', 0, 3, 5, 10, 20
Default: 'auto'
- 'model_tolerance':
-
The parameter specifies the tolerance of the projected 3D object model edges in the image, given in pixels. The higher the value is chosen, the fewer views need to be generated. Consequently, a higher value results in models that are less memory consuming and faster to find with
find_shape_model_3d. On the other hand, if the value is chosen too high, the robustness of the matching will decrease. Therefore, this parameter should only be modified with care. For most applications, a good compromise between speed and robustness is obtained when setting 'model_tolerance' to 1.Suggested values: 0, 1, 2
Default: 1
- 'union_adjacent_contours':
-
This parameter specifies if adjacent projected contours should be joined by the operator
project_shape_model_3dor not. Activating this option is equivalent to callingunion_adjacent_contours_xldafterwards, but significantly faster.List of values: 'true', 'false'
Default: 'false'
If the system variable (see set_system)
'opengl_hidden_surface_removal_enable' is set to 'true'
(which is default if it is available) the graphics card is used to accelerate
the computation of the visible faces in the model views. Depending on the
graphics card this is significantly faster than the analytic visibility
computation.
If 'fast_pose_refinement' is set to 'true', the
precomputations necessary for the pose refinement step in
find_shape_model_3d are also performed on the graphics card.
Be aware that the results of the OpenGL projection are slightly different
compared to the analytic projection.
Execution Information
- Multithreading type: reentrant (runs in parallel with non-exclusive operators).
- Multithreading scope: global (may be called from any thread).
- Automatically parallelized on internal data level.
This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.
Parameters
ObjectModel3D (input_control) object_model_3d → (handle)
Handle of the 3D object model.
CamParam (input_control) campar → (real / integer / string)
Internal camera parameters.
RefRotX (input_control) angle.rad → (real)
Reference orientation: Rotation around x-axis or x component of the Rodriguez vector (in radians or without unit).
Default: 0
Suggested values: -1.57, -0.78, -0.17, 0., 0.17, 0.78, 1.57
RefRotY (input_control) angle.rad → (real)
Reference orientation: Rotation around y-axis or y component of the Rodriguez vector (in radians or without unit).
Default: 0
Suggested values: -1.57, -0.78, -0.17, 0., 0.17, 0.78, 1.57
RefRotZ (input_control) angle.rad → (real)
Reference orientation: Rotation around z-axis or z component of the Rodriguez vector (in radians or without unit).
Default: 0
Suggested values: -1.57, -0.78, -0.17, 0., 0.17, 0.78, 1.57
OrderOfRotation (input_control) string → (string)
Meaning of the rotation values of the reference orientation.
Default: 'gba'
List of values: 'abg', 'gba', 'rodriguez'
LongitudeMin (input_control) angle.rad → (real)
Minimum longitude of the model views.
Default: -0.35
Suggested values: -0.78, -0.35, -0.17
LongitudeMax (input_control) angle.rad → (real)
Maximum longitude of the model views.
Default: 0.35
Suggested values: 0.17, 0.35, 0.78
Restriction:
LongitudeMax >= LongitudeMin
LatitudeMin (input_control) angle.rad → (real)
Minimum latitude of the model views.
Default: -0.35
Suggested values: -0.78, -0.35, -0.17
Restriction:
- pi / 2 <= LatitudeMin && LatitudeMin <= pi / 2
LatitudeMax (input_control) angle.rad → (real)
Maximum latitude of the model views.
Default: 0.35
Suggested values: 0.17, 0.35, 0.78
Restriction:
- pi / 2 <= LatitudeMax && LatitudeMax <= pi / 2 && LatitudeMax >= LatitudeMin
CamRollMin (input_control) angle.rad → (real)
Minimum camera roll angle of the model views.
Default: -3.1416
Suggested values: -3.14, -1.57, -0.39, 0.0, 0.39, 1.57, 3.14
CamRollMax (input_control) angle.rad → (real)
Maximum camera roll angle of the model views.
Default: 3.1416
Suggested values: -3.14, -1.57, -0.39, 0.0, 0.39, 1.57, 3.14
Restriction:
CamRollMax >= CamRollMin
DistMin (input_control) number → (real)
Minimum camera-object-distance of the model views.
Default: 0.3
Suggested values: 0.05, 0.1, 0.2, 0.5
Restriction:
DistMin > 0
DistMax (input_control) number → (real)
Maximum camera-object-distance of the model views.
Default: 0.4
Suggested values: 0.1, 0.2, 0.5, 1.0
Restriction:
DistMax >= DistMin
MinContrast (input_control) number → (integer)
Minimum contrast of the objects in the search images.
Default: 10
Suggested values: 1, 2, 3, 5, 7, 10, 20, 30, 1000, 2000, 5000
GenParamName (input_control) attribute.name(-array) → (string)
Names of (optional) parameters for controlling the behavior of the operator.
Default: []
List of values: 'fast_pose_refinement', 'lowest_model_level', 'metric', 'min_face_angle', 'min_size', 'model_tolerance', 'num_levels', 'optimization', 'part_size', 'union_adjacent_contours'
GenParamValue (input_control) attribute.name(-array) → (integer / real / string)
Values of the optional generic parameters.
Default: []
Suggested values: 0, 1, 2, 3, 4, 6, 8, 10, 'auto', 'none', 'point_reduction_low', 'point_reduction_medium', 'point_reduction_high', 0.1, 0.2, 0.3, 'ignore_local_polarity', 'ignore_part_polarity', 'true', 'false'
ShapeModel3DID (output_control) shape_model_3d → (handle)
Handle of the 3D shape model.
Result
If the parameters are valid, the operator
create_shape_model_3d returns the value 2 (
H_MSG_TRUE)
. If necessary
an exception is raised. If the parameters are chosen such that all
model views contain too few points, the error 8510 is raised. In the
case that the projected model is bigger than twice the image size in
at least one model view, the error 8910 is raised.
Possible Predecessors
read_object_model_3d,
project_object_model_3d,
get_object_model_3d_params
Possible Successors
find_shape_model_3d,
write_shape_model_3d,
project_shape_model_3d,
get_shape_model_3d_params,
get_shape_model_3d_contours
See also
convert_point_3d_cart_to_spher,
convert_point_3d_spher_to_cart,
create_cam_pose_look_at_point,
trans_pose_shape_model_3d
References
Markus Ulrich, Christian Wiedemann, Carsten Steger, “Combining Scale-Space and Similarity-Based Aspect Graphs for Fast 3D Object Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1902-1914, Oct., 2012.
Module
3D Metrology