Workflow for semantic segmentationđź”—
We assume that your dataset is already labeled; see also About data.
Tip
Take a look at the HDevelop example series segment_pill_defects_deep_learning for an application.
The example segment_edges_deep_learning_with_retraining shows the complete workflow for an edge extraction application.
-
Preprocess the data.
-
The information about what is found on each image of your training dataset needs to be transferred. This is done by the procedure
read_dl_dataset_segmentation.
The dictionary
DLDatasetserves as a database storing all necessary information about your data. For more information about the data and the way it is transferred, see About data below and the page HALCON DL model. -
Split the dataset represented by the dictionary
DLDataset. This can be done using the proceduresplit_dl_dataset.
The resulting split will be saved via the key
splitin each sample entry ofDLDataset. -
Now you can preprocess your dataset. For this, you can use the procedure
preprocess_dl_dataset.
This procedure also offers guidance on how to implement a customized preprocessing procedure.
To use this procedure, specify the preprocessing parameters as for example, the image size. For the latter, you should select the smallest possible image size at which the regions to be segmented can still be clearly recognised. Store all the parameter with their values in a dictionaryDLPreprocessParam. You can use the procedurecreate_dl_preprocess_param.
We recommend saving this dictionary
DLPreprocessParamto have access to the preprocessing parameter values later during the inference phase.During the preprocessing of your dataset also the images
weight_imagewill be generated for the training dataset bypreprocess_dl_dataset. They assign to each class the weight (“class weights”) that its pixels receive during training (see Model parameters and hyperparameters below).
These steps are also shown in the HDevelop example
segment_pill_defects_deep_learning_1_preprocess.hdev. -
-
Train the model.
-
Read a network using the operator
-
Set the model parameters via the operator
Such parameters are, for example,
'image_dimensions'and'class_ids'. For more information, seeget_dl_model_param.Tip
You can always retrieve the current parameter values using the operator
get_dl_model_param. -
Set the training parameters and store them in the dictionary
TrainParam. These parameters include:- Hyperparameters, as described in Model parameters and hyperparameters below and in the deep learning overview page
- Parameters for possible data augmentation (optional)
- Parameters for the evaluation during training
- Parameters for the visualization of training results
- Parameters for serialization
This can be done using the procedure
create_dl_train_param.
-
Train the model. This can be done using the procedure
train_dl_model.
The procedure expects:
- Model handle
DLModelHandle - Dictionary with the data information
DLDataset - Dictionary with the training parameter
TrainParam - Information how many epochs the training shall run
If you use the procedure
train_dl_model, the total loss as well as optional evaluation measures are visualized.
These steps are also shown in the HDevelop example
segment_pill_defects_deep_learning_2_train.hdev. -
-
Evaluate the trained model.
-
Set the model parameters that may influence the evaluation, for example the
'batch_size', using the operator -
The evaluation can conveniently be done using the procedure
evaluate_dl_model.
-
The dictionary
EvaluationResultholds the requested evaluation measures. You can visualize your evaluation results using the proceduredev_display_segmentation_evaluation.
These steps are also shown in the HDevelop example
segment_pill_defects_deep_learning_3_evaluate.hdev. -
-
Run inference on new images.
-
Set the parameters, for example the
'batch_size', using the operator -
Generate a data dictionary
DLSamplefor each image. This can be done using the proceduregen_dl_samples_from_images.
-
Preprocess the image as done for the training. We recommend using the procedure
preprocess_dl_samples.
If you have saved the dictionary
DLPreprocessParamduring the preprocessing step, you can directly use it as input to specify all parameter values. -
Apply the model using the operator
-
Retrieve the results from the dictionary
'DLResultBatch'. The regions of the particular classes can be selected using for example, the operatorthresholdon the segmentation image.
These steps are also shown in the HDevelop example
segment_pill_defects_deep_learning_4_infer.hdev. -