General anomaly detection workflow🔗
The general workflow for an anomaly detection or Global Context Anomaly Detection task based on deep learning is as follows:
-
Preprocess the data.
-
The information content of your dataset needs to be converted. This can be done using the procedure
read_dl_dataset_anomaly.
It creates a dictionary
DLDataset, which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see About data and the page About the HALCON deep learning model. -
Split the dataset represented by the dictionary
DLDataset. This can be done using the proceduresplit_dl_dataset.
-
The network imposes several requirements on the images. To retrieve these requirements (for example, the image size and gray value range), read the model by using
followed by
-
Now you can preprocess your dataset. For this, you can use the procedure
preprocess_dl_dataset.
In case of custom preprocessing, this procedure offers guidance on the implementation.
To use this procedure, specify the preprocessing parameters as, for example, the image size. Store all the parameter with their values in a dictionary
DLPreprocessParam, for which you can use the procedurecreate_dl_preprocess_param.
We recommend saving this dictionary
DLPreprocessParamto have access to the preprocessing parameter values later during the inference phase.
-
-
Train the model.
-
Set the training parameters and store them in the dictionary
TrainParam. This can be done using the procedurecreate_dl_train_param.
-
Train the model. This can be done using the procedure
train_dl_model.
The procedure does the following:
- Adapts models of type
'gc_anomaly_detection'to the image statistics of the dataset calling the procedurenormalize_dl_gc_anomaly_features - Calls the corresponding training operator
train_dl_model_anomaly_dataset('anomaly_detection') ortrain_dl_model_batch('gc_anomaly_detection'), respectively
The procedure expects the following:
- Model handle
DLModelHandle< - Dictionary
DLDatasetcontaining the data information - Dictionary
TrainParamcontaining the training parameters
-
Normalize the network.
This step is only necessary when using a Global Context Anomaly Detection model.
The anomaly scores need to be normalized by applying the procedurenormalize_dl_gc_anomaly_scores.
This needs to be done to get reasonable results when applying a threshold on the anomaly scores later (see Specific parameters).
-
-
Evaluate the trained model.
-
Set the model parameters which may influence the evaluation.
-
The evaluation can be done conveniently using the procedure
evaluate_dl_model.
This procedure expects a dictionary
GenParamwith the evaluation parameters. -
The dictionary
EvaluationResultholds the desired evaluation measures.
-
-
Run inference on new images.
For a trained model, perform the following steps:
-
Request the requirements the model imposes on the images using the operator
or the procedure
create_dl_preprocess_param_from_model.
-
Set the model parameter described in Specific parameters, using the operator
-
Generate a data dictionary
DLSamplefor each image. This can be done using the proceduregen_dl_samples_from_images.
-
Every image has to be preprocessed the same way as for the training. For this, you can use the procedure
preprocess_dl_samples.
When you saved the dictionary
DLPreprocessParamduring the preprocessing step, you can directly use it as input to specify all parameter values. -
Apply the model using the operator
-
Retrieve the results from the dictionary
DLResult.
-