Skip to content

Workflow for semantic segmentationđź”—

We assume that your dataset is already labeled; see also About data.

Tip

Take a look at the HDevelop example series segment_pill_defects_deep_learning for an application. The example segment_edges_deep_learning_with_retraining shows the complete workflow for an edge extraction application.

  1. Preprocess the data.

    1. The information about what is found on each image of your training dataset needs to be transferred. This is done by the procedure

      • read_dl_dataset_segmentation.

      The dictionary DLDataset serves as a database storing all necessary information about your data. For more information about the data and the way it is transferred, see About data below and the page HALCON DL model.

    2. Split the dataset represented by the dictionary DLDataset. This can be done using the procedure

      • split_dl_dataset.

      The resulting split will be saved via the key split in each sample entry of DLDataset.

    3. Now you can preprocess your dataset. For this, you can use the procedure

      • preprocess_dl_dataset.

      This procedure also offers guidance on how to implement a customized preprocessing procedure.
      To use this procedure, specify the preprocessing parameters as for example, the image size. For the latter, you should select the smallest possible image size at which the regions to be segmented can still be clearly recognised. Store all the parameter with their values in a dictionary DLPreprocessParam. You can use the procedure

      • create_dl_preprocess_param.

      We recommend saving this dictionary DLPreprocessParam to have access to the preprocessing parameter values later during the inference phase.

      During the preprocessing of your dataset also the images weight_image will be generated for the training dataset by preprocess_dl_dataset. They assign to each class the weight (“class weights”) that its pixels receive during training (see Model parameters and hyperparameters below).

    These steps are also shown in the HDevelop example segment_pill_defects_deep_learning_1_preprocess.hdev.

  2. Train the model.

    1. Read a network using the operator

    2. Set the model parameters via the operator

      Such parameters are, for example, 'image_dimensions' and 'class_ids'. For more information, see get_dl_model_param.

      Tip

      You can always retrieve the current parameter values using the operator get_dl_model_param.

    3. Set the training parameters and store them in the dictionary TrainParam. These parameters include:

      This can be done using the procedure

      • create_dl_train_param.
    4. Train the model. This can be done using the procedure

      • train_dl_model.

      The procedure expects:

      • Model handle DLModelHandle
      • Dictionary with the data information DLDataset
      • Dictionary with the training parameter TrainParam
      • Information how many epochs the training shall run

      If you use the procedure train_dl_model, the total loss as well as optional evaluation measures are visualized.

    These steps are also shown in the HDevelop example segment_pill_defects_deep_learning_2_train.hdev.

  3. Evaluate the trained model.

    1. Set the model parameters that may influence the evaluation, for example the 'batch_size', using the operator

    2. The evaluation can conveniently be done using the procedure

      • evaluate_dl_model.
    3. The dictionary EvaluationResult holds the requested evaluation measures. You can visualize your evaluation results using the procedure

      • dev_display_segmentation_evaluation.

    These steps are also shown in the HDevelop example segment_pill_defects_deep_learning_3_evaluate.hdev.

  4. Run inference on new images.

    1. Set the parameters, for example the 'batch_size', using the operator

    2. Generate a data dictionary DLSample for each image. This can be done using the procedure

      • gen_dl_samples_from_images.
    3. Preprocess the image as done for the training. We recommend using the procedure

      • preprocess_dl_samples.

      If you have saved the dictionary DLPreprocessParam during the preprocessing step, you can directly use it as input to specify all parameter values.

    4. Apply the model using the operator

    5. Retrieve the results from the dictionary 'DLResultBatch'. The regions of the particular classes can be selected using for example, the operator threshold on the segmentation image.

    These steps are also shown in the HDevelop example segment_pill_defects_deep_learning_4_infer.hdev.