Workflow for instance segmentation and object detection🔗
We assume that your dataset is already labeled; see also About data.
Tip
Take a look at the HDevelop example series detect_pills_deep_learning for an application.
The example dl_instance_segmentation_workflow.hdev shows the complete workflow for an instance segmentation application.
-
Create the model and preprocess the dataset.
-
Create a model using the operator
⤷ The operator returns a handle
'DLModelHandle'.You have to specify at least the backbone and the number of classes to be distinguished. For instance segmentation, set the parameter
'instance_segmentation'to create an according model.Further parameters can be set via the dictionary
DLModelDetectionParam. Their values should be well chosen for the specific task to help reduce memory consumption and runtime. See the operator documentation for more information.Anchor parameters suiting your dataset can be estimated using the procedure
determine_dl_model_detection_param.
In case the dataset is not representing all orientations the network will face during training (for example, due to augmentation), the suggested values need to be adapted accordingly.
Note
After creating the model, its underlying network architecture is fixed to the specified input values.
Alternatively, to read in a model you have already saved with
write_dl_model, you can also use -
The information about what is found on each image of your training dataset needs to be read in. This can be done by reading the data out of one of the following:
-
DLDatasetdictionary usingread_dict -
File in the COCO data format using the procedure
read_dl_dataset_from_coco, which creates a dictionaryDLDataset
The dictionary
DLDatasetserves as a database storing all necessary information about your data. For more information about the data and the way it is transferred, see About data below and the page HALCON DL model. -
-
Split the dataset represented by the dictionary
DLDataset. This can be done using the proceduresplit_dl_dataset.
⤷ The resulting split is saved via the key
splitin each sample entry ofDLDataset. -
The network imposes requirements on the images, for example, the image width and height. You can retrieve every single value using the operator
Alternatively, you can retrieve all necessary parameters using the procedure
create_dl_preprocess_param_from_model.
Important
For classes declared by
'class_ids_no_orientation', the bounding boxes need to be treated specially during preprocessing. As a consequence, these classes should be set at this point latest.Now you can preprocess your dataset. For this, you can use the procedure
preprocess_dl_dataset.
This procedure also offers guidance on how to implement a customized preprocessing procedure. We recommend preprocessing and storing all images used for the training before starting the training, as this speeds up the training significantly.
To visualize the preprocessed data, you can use the procedure
dev_display_dl_data.
These steps are also shown in the HDevelop example
detect_pills_deep_learning_1_prepare.hdev. -
-
Train the model.
-
Set the training parameters and store them in the dictionary
TrainParam. These parameters include:- Hyperparameters, as described in Model parameters and hyperparameters and in the deep learning overview page
- Parameters for possible data augmentation (optional)
-
Train the model. This can be done using the procedure
train_dl_model.
The procedure expects the following:
- Model handle
DLModelHandle - Dictionary with the data information
DLDataset - Dictionary with the training parameters
TrainParam - Information how many epochs the training shall run
⤷ During the training, the total loss minimizes.
These steps are also shown in the HDevelop example
detect_pills_deep_learning_2_train.hdev. -
-
Evaluate the trained model.
-
Set the model parameters that may influence the evaluation.
-
The evaluation can conveniently be done using the procedure
evaluate_dl_model.
This procedure expects a dictionary
GenParamEvalwith the evaluation parameters.
To get the data necessary for visualization, set the parameterdetailed_evaluationto'true'. -
You can visualize your evaluation results using the procedure
dev_display_detection_detailed_evaluation.
These steps are also shown in the HDevelop example
detect_pills_deep_learning_3_evaluate.hdev. -
-
Run inference on new images.
-
Request the requirements the network imposes on the images using the operator
or the procedure
create_dl_preprocess_param_from_model.
-
Set the model parameter described in Model parameters and hyperparameters using the operator
The
'batch_size'can be generally set independently from the number of images to be inferred. For more information about how to set this parameter for greater efficiency, seeapply_dl_model. -
Generate a data dictionary
DLSamplefor each image. This can be done using the proceduregen_dl_samples_from_images.
-
Every image has to be preprocessed as done for the training. For this, you can use the procedure
preprocess_dl_samples.
-
Apply the model using the operator
-
Retrieve the results from the dictionary
'DLResultBatch'.
These steps are also shown in the HDevelop example
detect_pills_deep_learning_4_infer.hdev. -