Skip to content

Workflow for Deep OCR🔗

Basic workflow🔗

Tip

Take a look at the HDevelop example deep_ocr_workflow.hdev for an application.

  1. Create the Deep OCR model.

    Create a Deep OCR model containing either one or both of the two model components detection_model and recognition_model using the operator

    To use a retrained model component instead of the provided one, adjust the created model by setting the retrained model component as 'recognition_model' or 'detection_model' using set_deep_ocr_param.

  2. Run inference on images.

    Set model parameters regarding, for example, the used devices, image dimensions, or minimum scores using set_deep_ocr_param.

    Apply the Deep OCR model on your acquired images using

    Note

    The inference results depend on the used model components. For information about which dictionary entries are computed for each model composite, see apply_deep_ocr.

    The inference results can be retrieved from the dictionary DeepOCRResult. The following procedures are provided to visualize results and score maps:

    • Show location and/or recognized word using dev_display_deep_ocr_results.

    • Show location (and, if inferred, recognized word) on preprocessed image using dev_display_deep_ocr_results_preprocessed (if the model contains detection_model).

    • Show score maps using dev_display_deep_ocr_score_maps (if the model contains detection_model).

Training and evaluating the model components🔗

Tip

Take a look at the HDevelop examples deep_ocr_recognition_training_workflow.hdev or deep_ocr_detection_training_workflow.hdev for an application.

  1. Preprocess the data.

    1. The information to be obtained from the images of your training dataset needs to be transferred. This is done by the procedure

      • read_dl_dataset_ocr_recognition for the recognition component of a Deep OCR model

      • read_dl_dataset_ocr_detection for the detection component of a Deep OCR model

      It creates a dictionary DLDataset, which serves as a database and stores all necessary information about your data. For more information about the data and the way it is transferred, see About data and the page Deep learning > HALCON DL model.

    2. Split the dataset represented by the dictionary DLDataset. This can be done using the procedure

      • split_dl_dataset.
    3. The network imposes several requirements on the images. To retrieve these requirements (for example, the image size and gray value range), read the model by using

      followed by

    4. Now you can preprocess your dataset. For this, you can use the procedure

      • preprocess_dl_dataset.

      To use this procedure, specify the preprocessing parameters, for example, the image size. Store all the parameter with their values in a dictionary DLPreprocessParam, for which you can use the procedure

      • create_dl_preprocess_param_from_model.

      We recommend saving this dictionary DLPreprocessParam to have access to the preprocessing parameter values later during the inference phase.

  2. Train the model.

    1. Set the training parameters and store them in the dictionary TrainParam. This can be done using the procedure

      • create_dl_train_param.
    2. Train the model. This can be done using the procedure

      • train_dl_model.

      The procedure expects the following:

      • Model handle DLModelHandle
      • Dictionary DLDataset containing the data information
      • Dictionary TrainParam containing the training parameters
  3. Evaluate the retrained model.

    1. Set the model parameters which may influence the evaluation.

    2. The evaluation can be done conveniently using the procedure

      • evaluate_dl_model.

      This procedure expects a dictionary GenParamEval with the evaluation parameters.

    3. The dictionary EvaluationResult holds the evaluation measures.

      To get a sense of how the retrained model performed against the pretrained model, you can compare their evaluation values. To understand the different evaluation measures, see Evaluation measures for Deep OCR results below.

    After a successful evaluation, the retrained model can be used for inference as described in About the general workflow above.

Programming examples🔗

Some example applications are shown in the following programs:

  • Locating and recognizing text:
    %HALCONEXAMPLES%/hdevelop/OCR/Deep-OCR/deep_ocr_workflow.hdev

  • Recognizing text without detection:
    %HALCONEXAMPLES%/hdevelop/OCR/Deep-OCR/deep_ocr_workflow.hdev

  • Retraining the recognition component:
    %HALCONEXAMPLES%/hdevelop/OCR/Deep-OCR/deep_ocr_recognition_training_workflow.hdev

  • Retraining the detection component:
    %HALCONEXAMPLES%/hdevelop/OCR/Deep-OCR/deep_ocr_detection_training_workflow.hdev

  • Handling large images:
    %HALCONEXAMPLES%/hdevelop/OCR/Deep-OCR/deep_ocr_workflow.hdev