Skip to content

MLP classification🔗

Multi-layer perceptrons (MLP) are neural-network-based classifiers that learn separating hyperplanes directly from labeled training data. During training, the network adjusts its weights such that feature vectors of different classes are separated in the feature space.

For two classes, the separating hyperplane divides the feature space into two regions. Feature vectors on one side of the hyperplane are assigned to one class, while feature vectors on the other side are assigned to the other class. For multiple classes, the hyperplanes are chosen such that the feature vectors of the correct class have the largest positive distance from the corresponding hyperplane.

A single-layer neural network can be used for classification if the classes are linearly separable. As illustrated in Figure 1, a single-layer network can be used for both two-class classification (a) and multi-class classification (b).

For classes that are not linearly separable, a hidden layer can be introduced to model non-linear decision boundaries. The multi-layer network shown in (c) consists of an input layer, a hidden layer, and an output layer.

Figure 1: Single-layer and multi-layer neural networks.

In HALCON, the implemented MLP classifier uses a single hidden layer. A sufficient number of processing units in this layer allows the network to approximate complex separating hypersurfaces and output functions with values in the range [0, 1].

Processing units🔗

Each processing unit first computes a linear combination of its input values and the corresponding network weights. A nonlinear activation function is then applied to the resulting activation.

The inputs are either the feature vector or the outputs of the previous network layer.

Figure 2: Processing unit of an MLP

The activation of a processing unit is computed as

\[ a_j^{(l)} = \sum_{i=1}^{n_l} w_{ij}^{(l)} x_i^{(l-1)} + b_j^{(l)} \]

where

  • \(x_i^{(l-1)}\) are the outputs of the previous layer,
  • \(w_{ij}^{(l)}\) are the network weights,
  • \(b_j^{(l)}\) is the bias term, and
  • \(a_j^{(l)}\) is the activation of the processing unit.

The activation is then transformed by a nonlinear activation function:

\[ x_j^{(l)} = f(a_j^{(l)}) \]

For the hidden layer, HALCON uses the hyperbolic tangent function

\[ f(x)=\tanh(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}. \]

For classification, the output layer uses the softmax activation function

\[ f(x_i)=\frac{e^{x_i}}{\sum_{j=1}^{n}e^{x_j}}, \]

which maps the output values to the range (0, 1) such that they add up to 1.

Training🔗

During training, the network weights are optimized using labeled training samples. Each training sample consists of a feature vector and its corresponding class label.

The feature vector is propagated through the network and the resulting output is compared with the expected output. If the output does not correspond to the expected class within the specified error tolerance, the network weights are incrementally adjusted to reduce the classification error.

HALCON uses a stable numerical algorithm for adjusting the network weights rather than the classical backpropagation algorithm.

The quality of the trained classifier strongly depends on the representativeness of the training data. Therefore, the training samples should cover the expected variations within each class while avoiding irrelevant variations that may reduce the generalization capability of the classifier.

Further information on parameter optimization can be found in the Optimization chapter.

Typical applications🔗

MLP classifiers can be applied to a wide range of classification tasks. Typical applications include

  • general feature classification,
  • image segmentation, and
  • optical character recognition (OCR).

MLP can also be used for least-squares fitting (regression) and for classification problems with multiple independent logical attributes.

An MLP can have more than one hidden layer and is then considered a deep learning method. In HALCON, however, only a single hidden layer is implemented in the MLP classifier. Therefore, the MLP method is not considered a deep learning method in this context.