Skip to content

Optimization for MLP-based classification for OCR๐Ÿ”—

Adjusting creation of MLP-based OCR classifiers๐Ÿ”—

An MLP classifier for OCR is created using create_ocr_class_mlp. Here, most of the important parameters have to be set.

Like for the general classification, OCR uses a set of features to classify regions into classes, which in this case correspond to specific characters and numbers. Some of the features that can be used for OCR are gray value features for which the number of returned features varies dependent on the regionโ€™s size. As a classifier requires a constant number of features, i.e., the dimension of the feature vector has to be the same for all training samples and all regions to be classified, the region of a character has to be transformed (scaled) to a standard size. This size is determined by WidthCharacter and HeightCharacter. In most applications, sizes between 6x8 and 10x14 should be used. As a rule of thumb, the values may be small if only few characters have to be distinguished but large when classifying characters with complex shapes (e.g., Japanese signs).

The input parameter Interpolation is needed to control the transformation of the region to the size specified with WidthCharacter and HeightCharacter. Generally, when transforming a region, transformed points will lie between discrete pixel coordinates. To assign each point to its final pixel coordinate and additionally to avoid aliasing, which typically occurs for scaled regions, an appropriate interpolation scheme is needed. Three types of interpolation with different quality and speed properties are provided by HALCON and can be selected for the parameter Interpolation:

  • nearest_neighbor

    If nearest_neighbor is selected, a nearest-neighbor interpolation is applied. There, the gray value is determined from the nearest pixelโ€™s gray value. This interpolation scheme is very fast but may lead to a low interpolation quality.

  • bilinear

    If bilinear is selected, a bilinear interpolation is applied. There, the gray value is determined from the four nearest pixels through bilinear interpolation. This interpolation scheme is of medium speed and quality. Do not use it if the characters in the image appear larger than WidthCharacter times HeightCharacter. In this case, the interpolation method constant or weighted should be used.

  • constant

    If constant is selected, a bilinear interpolation is applied. There, the gray value is determined from the four nearest pixels through bilinear interpolation. If the transformation contains a scaling with a scale factor smaller than 1, a kind of mean filter is used to prevent aliasing effects. This interpolation scheme is of medium speed and quality.

  • weighted

    If weighted is selected, again a bilinear interpolation is applied, but now, aliasing effects for a scaling with a scale factor smaller than 1 are prevented by a kind of Gaussian filter instead of a mean filter. This interpolation scheme is slow but leads to the best quality.

The interpolation should be chosen such that no aliasing effects occur in the transformation. For most applications, Interpolation should be set to constant.

The input parameter Features specifies the features that are used for the classification. Features can contain a tuple of several feature names. Each of these feature names results in one or more features to be calculated for the classifier. That is, the length of the tuple in Features is similar or smaller than the dimension of the final feature vector. In the reference of create_ocr_class_mlp all features that are available for OCR are listed. To classify characters, in most cases the default setting can be used. Then, the features ratio and pixel_invar are selected. Note that the selection of the features significantly influences the quality of the classification.

The input parameter Characters contains a tuple with the names of the characters that will be trained. Each name must be passed as a string. The number of elements in the tuple determines the number of available classes.

The input parameter NumHidden follows the same rules as provided for the corresponding operator used to create an MLP classifier for a general classification.

The parameters Preprocessing and NumComponents follow the same rules as provided for the corresponding operator used to create an MLP classifier for a general classification. The only exception is that for the OCR classification the features are already approximately normalized. Thus, Preprocessing can typically be set to none.

If Preprocessing is set to principal_components or canonical_variates you can use the operator get_prep_info_ocr_class_mlp to determine the optimum number of components.

The parameter RandSeed follows the same rules as provided for the corresponding operator used to create an MLP classifier for a general classification.

The output parameter OCRHandle is the handle of the classifier that is needed and modified throughout the following classification steps.

Adjusting storage in training file๐Ÿ”—

After creating a classifier and segmenting the regions for the characters of known classes, i.e., character names, the samples must be stored to a training file. Here, the same operators are used for MLP and SVM classifiers.

Different operators are provided for storing training samples to file. You can either store all samples to file in one step by inserting a tuple containing all regions and a tuple containing all corresponding class names to the operator write_ocr_trainf. Or you can successively append single training samples to the training file using append_ocr_trainf. If you choose the latter, be aware that the training file is extended every time you run the program, i.e., it is not created anew. So, if you want to successively add samples, it is recommended to use write_ocr_trainf for the first sample and append_ocr_trainf for all following samples. The operators are applied as follows:

  • Store training samples to a new training file:

    When storing all training samples simultaneously into a file using write_ocr_trainf, you have to assign a tuple of regions that represent characters to the parameter Character. The image that contains the regions must be set in Image so that knowledge about the gray values within the regions is available. In Class a tuple of class names that correspond to the regions with the same tuple index must be inserted. Finally, you specify the name and path of the stored training file in FileName.

  • Append training samples to a training file:

    When successively storing individual training samples into a file using append_ocr_trainf, the same parameters as for write_ocr_trainf have to be set. But in contrast to the operator write_ocr_trainf the characters are appended to an existing file using the same training file format. If the file does not exist, a new file is generated.

If no file extension is specified in FileName the extension .trf is appended to the file name. The version of the file format used for writing data can be defined by the parameter ocr_trainf_version of the operator set_system.

If you have several training files that you want to combine, you can concatenate them with the operator concat_ocr_trainf.

Adjusting training of MLP-based OCR classifiers๐Ÿ”—

trainf_ocr_class_mlp trains the OCR classifier OCRHandle with the training characters stored in the OCR training file given by TrainingFile. The remaining parameters MaxIterations, WeightTolerance, ErrorTolerance, Error, and ErrorLog have the same meaning as introduced for the training of an MLP classifier for a general classification.

Adjusting multi-class classification with MLP-based OCR classifiers๐Ÿ”—

With do_ocr_multi_class_mlp multiple characters can be classified in a single call. Typically, this is faster than successively applying do_ocr_single_class_mlp, which classifies single characters, in a loop. However, do_ocr_multi_class_mlp can only return the best class of each character. If the second best class is needed, e.g., because the classes significantly overlap for the possible outliers related to the confidence values of MLP classifications), do_ocr_single_class_cnn should be used instead. The following parameters have to be set for do_ocr_multi_class_mlp:

The input parameter Character contains a tuple of regions that have to be classified.

The input parameter Image contains the image that provides the gray value information for the regions that have to be classified.

The input parameter OCRHandle is the handle of the classifier that was trained with trainf_ocr_class_mlp.

The output parameter Class returns the result of the classification, i.e., a tuple of character names that correspond to the input regions that were given in the tuple Character. If the result is '\textbackslash 0x1A' or '\textbackslash 032', the respective region has been classified as rejection class.

The output parameter Confidence returns the confidence value for the classification. Note, that for the confidence of MLP classifications, outliers are possible.

Adjusting single-class classification with MLP-based OCR classifiers๐Ÿ”—

Instead of using do_ocr_multi_class_mlp to add all samples in a single call, do_ocr_single_class_mlp can be used to successively add samples. Then, besides the best class for a region, also the second best (and third best etc.) class can be obtained. This may be suitable, if the class membership of a region is uncertain because of, e.g., overlapping classes. If only the best class for each region is searched for, do_ocr_multi_class_mlp is faster and therefore recommended.

The input parameter Character contains a single region that has to be classified.

The input parameter Image contains the image that provides the gray value information for the region that has to be classified.

The input parameter OCRHandle is the handle of the classifier that was trained with trainf_ocr_class_mlp.

The input parameter Num specifies the number of best classes to be searched for. Generally, Num is set to 1 if only the class with the best probability is searched for, and to 2 if the second best class is also of interest, e.g., because the classes overlap.

The output parameter Class returns the result of the classification, i.e., the Num best character names that correspond to the input region that was specified in Character. If the result is '\textbackslash 0x1A' or '\textbackslash 032', the respective region has been classified as rejection class.

The output parameter Confidence returns the Num best confidence values for the classification. Note, that for the confidence of MLP classifications, outliers are possible.