Skip to content

Workflow for multi-label classification🔗

We assume that your dataset is already labeled; see also About data below.

Tip

Take a look at the HDevelop example dl_multi_label_classification_workflow.hdev for an application.

  1. Create a multi-label classification model.

    The model is created based on a backbone model. The creation can be done with the procedure

    • create_dl_model_multi_label_classification.

    Thereby, a model DLModelHandle is created, where the backbone serves as feature extractor, to which the necessary layers for the training are added. Generally, it is possible to use any deep learning model as a backbone. However, depending on the complexity, we recommend using one of the classification models.
    For more information about the backbone and which one to use, see the procedure documentation of create_dl_model_multi_label_classification.

    create_dl_model_multi_label_classification uses a docking layer to attach the necessary layers for multi-label classification to the backbone classifier. The pretrained classifiers provided by HALCON have already specified docking layers. If you use a self-provided classifier as backbone, you have to specify it yourself.

    After the model has been created, save it using write_dl_model and subsequently read it using read_dl_model.

  2. Preprocess the data.

    1. The information about what is found in which image of your training dataset needs to be transferred. This is done by the procedure

      • read_dl_dataset_multi_label_classification.

      Thereby, a dictionary DLDataset is created, which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see About data below and the page Deep learning > HALCON DL model.

    2. Split the dataset represented by the dictionary DLDataset. This can be done using the procedure

      • split_dl_dataset.

      ⤷ The resulting split is saved via the key split in each sample entry of DLDataset.

    3. Now you can preprocess your dataset. For this, you can use the procedure

      • preprocess_dl_dataset.

      In case of custom preprocessing, this procedure offers guidance on the implementation.

      To use this procedure, specify the preprocessing parameters as, for example, the image size. Store all the parameters with their values in a dictionary DLPreprocessParam. To do this, you can use the procedure

      • create_dl_preprocess_param.

      We recommend saving this dictionary DLPreprocessParam to have access to the preprocessing parameter values later during the inference phase.

    These steps are also shown in the HDevelop example dl_multi_label_classification_workflow.hdev.

  3. Train the model.

    1. Set the training parameters and store them in the dictionary TrainParam. These parameters include:

      • Hyperparameters, as described in the deep learning overview page
      • Parameters for possible data augmentation (optional)
      • Parameters for the evaluation during training
      • Parameters for the visualization of training results
      • Parameters for serialization

      This can be done using the procedure

      • create_dl_train_param.
    2. Train the model. This can be done using the procedure

      • train_dl_model.

      The procedure expects the following:

      • Model handle DLModelHandle
      • Dictionary with the data information DLDataset
      • Dictionary with the training parameter TrainParam
      • Information how many epochs the training shall run

      If the procedure train_dl_model is used, the total loss as well as optional evaluation measures are visualized.

    These steps are also shown in the HDevelop example dl_multi_label_classification_workflow.hdev.

  4. Evaluate the trained model.

    1. The evaluation can conveniently be done using the procedure

      • evaluate_dl_model.
    2. The dictionary EvaluationResult holds the resulting evaluation measures. You can visualize your evaluation results using the procedure

      • dev_display_multi_label_classification_evaluation.

    These steps are also shown in the HDevelop example dl_multi_label_classification_workflow.hdev.

  5. Run inference on new images.

    1. Set the parameters as, for example, 'batch_size' using the operator

    2. Generate a data dictionary DLSample for each image. This can be done using the procedure

      • gen_dl_samples_from_images.
    3. Preprocess the images as done for the training. We recommend using the procedure

      • preprocess_dl_samples.

      IF you saved the dictionary DLPreprocessParam during the preprocessing step, you can directly use it as input to specify all parameter values.

    4. Apply the model using the operator

    5. Retrieve the results from the dictionary 'DLResultBatch'.

    These steps are also shown in the HDevelop example dl_multi_label_classification_workflow.hdev.