Skip to content

Workflow for 3D Gripping Point Detection🔗

Tip

Take a look at the HDevelop example 3d_gripping_point_detection_workflow.hdev for an application.

About the general inference workflow🔗

  1. Read the pretrained 3D Gripping Point Detection model by using

  2. Set the model parameter regarding, for example, the used devices or image dimensions using

  3. Generate a data dictionary DLSample for each 3D scene. This can be done using the procedure

    • gen_dl_samples_3d_gripping_point_detection,

    which can cope with different kinds of 3D data. For more information about the data requirements, see Data.

  4. Preprocess the data before the inference. For this, you can use the procedure

    • preprocess_dl_samples.

    The required preprocessing parameters can be generated from the model with

    • create_dl_preprocess_param_from_model

    or set manually using

    • create_dl_preprocess_param.

    Important

    The preprocessing of the data has significant impact on the inference. For more informaton, see 3D scenes.

  5. Apply the model using the operator

  6. Perform a post-processing step on the resulting DLResult to retrieve gripping points for your scene using the procedure

    • gen_dl_3d_gripping_points_and_poses.
  7. Visualize the 2D and 3D results using the procedure

    • dev_display_dl_data or

    • dev_display_dl_3d_data, respectively.

Training and evaluating the model🔗

This paragraph describes how the 3D Gripping Point Detection model can be retrained and evaluated using custom data. An application scenario can be seen in the HDevelop example 3d_gripping_point_detection_training_workflow.hdev.

  1. Preprocess the data:

    1. The information content of your dataset needs to be converted. This is done by the procedure

      • read_dl_dataset_3d_gripping_point_detection.

      It creates a dictionary DLDataset, which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see Data and the page Deep learning > HALCON DL model.

    2. Split the dataset represented by the dictionary DLDataset. This can be done using the procedure

      • split_dl_dataset.
    3. The network imposes several requirements on the images. These requirements (for example, the image size and gray value range) can be retrieved by first reading the model with

      and then using

    4. Now you can preprocess your dataset. For this, you can use the procedure

      • preprocess_dl_dataset.

      To use this procedure, specify the preprocessing parameters as, for example, the image size. Store all parameters with their values in a dictionary DLPreprocessParam, for which you can use the procedure

      • create_dl_preprocess_param_from_model.

      Tip

      Save the DLPreprocessParam dictionary to have access to the preprocessing parameter values later during the inference phase.

  2. Train the model:

    1. Set the training parameters and store them in the dictionary TrainParam. This can be done using the procedure

      • create_dl_train_param.
    2. Train the model. This can be done using the procedure

      • train_dl_model.

      The procedure expects

      • the model handle DLModelHandle,

      • the dictionary DLDataset containing the data information,

      • the dictionary TrainParam containing the training parameters.

  3. Evaluate the retrained model:

    1. Set the model parameters that may influence the evaluation.

    2. The evaluation can be done conveniently using the procedure

      • evaluate_dl_model.

      This procedure expects a dictionary GenParam with the evaluation parameters.

    3. The dictionary EvaluationResult holds the evaluation measures. To understand how the retrained model performed compared to the pretrained model, you can compare their evaluation values. For more information about the different evaluation measures, see Evaluation measures for 3D Gripping Point Detection results.