create_class_gmmCreateClassGmmCreateClassGmmcreate_class_gmmT_create_class_gmm🔗

Short description🔗

create_class_gmmCreateClassGmmCreateClassGmmcreate_class_gmmT_create_class_gmm — Create a Gaussian Mixture Model for classification

Signature🔗

create_class_gmm( integer NumDim, integer NumClasses, integer NumCenters, string CovarType, string Preprocessing, integer NumComponents, integer RandSeed, out class_gmm GMMHandle )

Description🔗

create_class_gmmCreateClassGmm creates a Gaussian Mixture Model (GMM) for classification. NumDimnumDimnum_dim specifies the number of dimensions of the feature space, NumClassesnumClassesnum_classes specifies the number of classes. A GMM consists of NumCentersnumCentersnum_centers Gaussian centers per class. NumCentersnumCentersnum_centers can not only be the exact number of centers to be used, but, depending on the number of parameters, can specify upper and lower bounds for the number of centers:

exactly one parameter: The parameter determines the exact number of centers to be used for all classes.
exactly two parameters: The first parameter determines the minimum number of centers, the second determines the maximum number of centers for all classes.
exactly \(2\cdot NumClasses\) parameters: Alternatingly every first parameter determines the minimum number of centers per class and every second parameters determines the maximum number of centers per class.

When upper and lower bounds are specified, the optimum number of centers will be determined with the help of the Minimum Message Length Criterion (MML). In general, we recommend to start the training with (too) many centers as maximum and the expected number of centers as minimum.

Each center is described by the parameters center \({\bf m}_{j}\), covariance matrix \({\bf C}_{j}\), and mixing coefficient \(P_{j}\). These parameters are calculated from the training data by means of the Expectation Maximization (EM) algorithm. A GMM can approximate an arbitrary probability density, provided that enough centers are being used. The covariance matrices \({\bf C}_{j}\) have the dimensions NumDimnumDimnum_dim x NumDimnumDimnum_dim (NumComponentsnumComponentsnum_components x NumComponentsnumComponentsnum_components if preprocessing is used) and are symmetric. Further constraints can be given by CovarTypecovarTypecovar_type:

For CovarTypecovarTypecovar_type \(=\) 'spherical'"spherical", \({\bf C}_{j}\) is a scalar multiple of the identity matrix \({\bf C}_{j} = s_{j}^{2} {\bf I}\). The center density function \(p({\bf x}|j)\) is

\[\begin{eqnarray*} p({\bf x}|j) = \frac{1}{{(2\pi s_{j}^2)^{d/2}}} \exp({-\frac{{\left\| {{\bf x} - {\bf m}_{j}} \right\|^2}}{{2s_{j}^2}}}) \end{eqnarray*}\]

For CovarTypecovarTypecovar_type \(=\) 'diag'"diag", \({\bf C}_{j}\) is a diagonal matrix \({\bf C}_{j} = diag ( s_{j,1}^{2},...,s_{j,d}^{2} )\). The center density function \(p({\bf x}|j)\) is

\[\begin{eqnarray*} p({\bf x}|j) = \frac{1}{{(2\pi \prod\limits_{i = 1}^d {s_{j,i} ^2} )^{d/2} }}\exp ( { - \sum\limits_{i = 1}^d {\frac{{( {x_{i} - m_{j,i}} )^2 }}{{2s_{j,i} ^2 }}} } ) \end{eqnarray*}\]

For CovarTypecovarTypecovar_type \(=\) 'full'"full", \({\bf C}_{j}\) is a positive definite matrix. The center density function \(p({\bf x}|j)\) is

\[\begin{eqnarray*} p({\bf x}|j) = \frac{1}{{(2\pi )^{d/2} | {{\bf C}_{j} } |^{\frac{1}{2}} }}\exp ( { - \frac{1}{2}({\bf x} - {\bf m}_{j} )^T {\bf C}^{-1} ({\bf x} - {\bf m}_{j} )} ) \end{eqnarray*}\]

The complexity of the calculations increases from CovarTypecovarTypecovar_type \(=\) 'spherical'"spherical" over CovarTypecovarTypecovar_type \(=\) 'diag'"diag" to CovarTypecovarTypecovar_type \(=\) 'full'"full". At the same time the flexibility of the centers increases. In general, 'spherical'"spherical" therefore needs higher values for NumCentersnumCentersnum_centers than 'full'"full".

The procedure to use GMM is as follows: First, a GMM is created by create_class_gmmCreateClassGmm. Then, training vectors are added by add_sample_class_gmmAddSampleClassGmm, afterwards they can be written to disk with write_samples_class_gmmWriteSamplesClassGmm. With train_class_gmmTrainClassGmm the classifier center parameters (defined above) are determined. Furthermore, they can be saved with write_class_gmmWriteClassGmm for later classifications.

From the mixing probabilities \(P_{j}\) and the center density function \(p({\bf x}|j)\), the probability density function p(x) can be calculated by:

\[\begin{eqnarray*} p(x)=\sum\limits_{j = 1}^{n_{comp}} P(j) p({\bf x}|j) \end{eqnarray*}\]

The probability density function p(x) can be evaluated with evaluate_class_gmmEvaluateClassGmm for a feature vector x. classify_class_gmmClassifyClassGmm sorts the p(x) and therefore discovers the most probable class of the feature vector.

The parameters Preprocessingpreprocessingpreprocessing and NumComponentsnumComponentsnum_components can be used to preprocess the training data and reduce its dimensions. These parameters are explained in the description of the operator create_class_mlpCreateClassMlp.

create_class_gmmCreateClassGmm initializes the coordinates of the centers with random numbers. To ensure that the results of training the classifier with train_class_gmmTrainClassGmm are reproducible, the seed value of the random number generator is passed in RandSeedrandSeedrand_seed.

Execution information🔗

Execution information

Multithreading type: reentrant (runs in parallel with non-exclusive operators).
Multithreading scope: global (may be called from any thread).
Processed without parallelization.

This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.

Parameters🔗

NumDimnumDimnum_dim (input_control) integer → (integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Number of dimensions of the feature space.

Default: 33
Suggested values: 1, 2, 3, 4, 5, 8, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 100
Restriction: NumDim >= 1

NumClassesnumClassesnum_classes (input_control) integer → (integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Number of classes of the GMM.

Default: 55
Suggested values: 1, 2, 3, 4, 5, 6, 7, 8, 9, 101, 2, 3, 4, 5, 6, 7, 8, 9, 10
Restriction: NumClasses >= 1

NumCentersnumCentersnum_centers (input_control) integer(-array) → (integer)HTuple (Hlong)HTuple (int / long)MaybeSequence[int]Htuple (Hlong)

Number of centers per class.

Default: 11
Suggested values: 1, 2, 3, 4, 5, 8, 10, 15, 20, 301, 2, 3, 4, 5, 8, 10, 15, 20, 30
Restriction: NumClasses >= 1

CovarTypecovarTypecovar_type (input_control) string → (string)HTuple (HString)HTuple (string)strHtuple (char*)

Type of the covariance matrices.

Default: 'spherical'"spherical"
List of values: 'diag', 'full', 'spherical'"diag", "full", "spherical"

Preprocessingpreprocessingpreprocessing (input_control) string → (string)HTuple (HString)HTuple (string)strHtuple (char*)

Type of preprocessing used to transform the feature vectors.

Default: 'normalization'"normalization"
List of values: 'canonical_variates', 'none', 'normalization', 'principal_components'

NumComponentsnumComponentsnum_components (input_control) integer → (integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Preprocessing parameter: Number of transformed features (ignored for Preprocessingpreprocessingpreprocessing \(=\) 'none'"none" and Preprocessingpreprocessingpreprocessing \(=\) 'normalization'"normalization").

Default: 1010
Suggested values: 1, 2, 3, 4, 5, 8, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 100
Restriction: NumComponents >= 1

RandSeedrandSeedrand_seed (input_control) integer → (integer)HTuple (Hlong)HTuple (int / long)intHtuple (Hlong)

Seed value of the random number generator that is used to initialize the GMM with random values.

Default: 4242

GMMHandleGMMHandlegmmhandle (output_control) class_gmm → (handle)HTuple (HHandle)HClassGmm, HTuple (IntPtr)HHandleHtuple (handle)

GMM handle.

Example🔗

(HDevelop)

* Classification with Gaussian Mixture Models
create_class_gmm (NumDim , NumClasses, [1,5], 'full', 'none',\
                  NumComponents, 42, GMMHandle)
* Add the training data
for J := 0 to NumData-1 by 1
    * Features := [...]
    * ClassID := [...]
    add_sample_class_gmm (GMMHandle, Features, ClassID, Randomize)
endfor
* Train the GMM
train_class_gmm (GMMHandle, 100, 0.001, 'training', 0.0001, Centers, Iter)
* Classify unknown data in 'Features'
classify_class_gmm (GMMHandle, Features, 1, ID, Prob, Density, KSigmaProb)

Result🔗

If the parameters are valid, the operator create_class_gmmCreateClassGmm returns the value 2 (H_MSG_TRUE). If necessary an exception is raised.

Combinations with other operators🔗

Combinations

Possible successors

add_sample_class_gmmAddSampleClassGmm, add_samples_image_class_gmmAddSamplesImageClassGmm

Alternatives

create_class_mlpCreateClassMlp, create_class_svmCreateClassSvm

References🔗

Christopher M. Bishop: “Neural Networks for Pattern Recognition”; Oxford University Press, Oxford; 1995.

Mario A.T. Figueiredo: ``Unsupervised Learning of Finite Mixture Models’‘; IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 3; March 2002.

Module🔗

Foundation