Operator Reference
train_class_svm (Operator)
train_class_svm — Train a support vector machine.
Signature
Description
train_class_svm trains the support vector machine (SVM)
given in SVMHandle. Before the SVM can be trained, the
training samples to be used for the training must be added to the
SVM using add_sample_class_svm or
read_samples_class_svm.
Technically, training an SVM means solving a convex quadratic
optimization problem. This implies that it can be assured that
training terminates after finite steps at the global optimum. In
order to recognize termination, the gradient of the function that is
optimized internally must fall below a threshold, which is set in
Epsilon. By default, a value of 0.001 should be
used for Epsilon since this yields the best results in
practice. A too big value leads to a too early termination and
might result in suboptimal solutions. With a too small value the
optimization requires a longer time, often without changing the
recognition rate significantly. Nevertheless,
if longer training times are possible, a smaller value than
0.001 might be chosen. There are two common reasons for
changing Epsilon: First, if you specified a very small value
for Nu when calling (create_class_svm),
e.g., Nu = 0.001, a smaller Epsilon might
significantly improve the recognition rate. A second case is the
determination of the optimal kernel function and its parametrization (e.g.,
the KernelParam-Nu pair for the RBF kernel) with the
computationally intensive n-fold cross validation. Here, choosing a
bigger Epsilon reduces the computational time without
changing the parameters of the optimal kernel that would be obtained
when using the default Epsilon. After the optimal
KernelParam-Nu pair is obtained, the final
training is conducted with a small Epsilon.
The duration of the training depends on the training data, in
particular on the number of resulting support vectors (SVs), and
Epsilon. It can lie between seconds and several hours.
It is therefore recommended to choose the SVM parameter Nu
in create_class_svm so that as few SVs as possible are
generated without decreasing the recognition rate. Special care
must be taken with the parameter Nu in
create_class_svm so that the optimization starts from a
feasible region. If too many training errors are chosen with a too
big Nu, an exception is raised. In this case, an
SVM with the same training data, but with smaller Nu must
be trained.
With the parameter TrainMode you can choose between different
training modes. Normally, you train an SVM without additional
information and TrainMode is set to 'default'. If
multiple SVMs for the same data set but with different kernels are
trained, subsequent training runs can reuse optimization results and
thus speedup the overall training time of all runs. For this mode,
in TrainMode a SVM handle of a previously trained SVM is
passed. Note that the SVM handle passed in SVMHandle and
the SVMHandle passed in TrainMode must have the same
training data, the same mode and the same number of classes (see
create_class_svm). The application for this training mode is
the evaluation of different kernel functions given the same training
set. In the literature this is referred to as alpha seeding.
With TrainMode = 'add_sv_to_train_set' it is
possible to append the support vectors that were generated by a
previous call of train_class_svm to the currently saved
training set. This mode has two typical application areas: First, it
is possible to gradually train a SVM. For this, the complete
training set is divided into disjunctive chunks. The first chunk is
trained normally using TrainMode =
'default'. Afterwards, the previous training set is removed
with clear_samples_class_svm, the next chunk is added with
add_sample_class_svm and trained with TrainMode =
'add_sv_to_train_set'. This is repeated until all chunks
are trained. This approach has the advantage that even huge training
data sets can be trained efficiently with respect to memory
consumption. A second application area for this mode is that a
general purpose classifier can be specialized by adding
characteristic training samples and then retraining it. Please note that
the preprocessing (as described in create_class_svm) is not
changed when training with TrainMode =
'add_sv_to_train_set'.
Execution Information
- Multithreading type: reentrant (runs in parallel with non-exclusive operators).
- Multithreading scope: global (may be called from any thread).
- Processed without parallelization.
This operator modifies the state of the following input parameter:
During execution of this operator, access to the value of this parameter must be synchronized if it is used across multiple threads.
Parameters
SVMHandle (input_control, state is modified) class_svm → (handle)
SVM handle.
Epsilon (input_control) real → (real)
Stop parameter for training.
Default: 0.001
Suggested values: 0.00001, 0.0001, 0.001, 0.01, 0.1
TrainMode (input_control) number → (string / integer)
Mode of training. For normal operation: 'default'. If SVs already included in the SVM should be used for training: 'add_sv_to_train_set'. For alpha seeding: the respective SVM handle.
Default: 'default'
List of values: 'add_sv_to_train_set', 'default'
Example (HDevelop)
* Train an SVM
create_class_svm (NumFeatures, 'rbf', 0.01, 0.01, NumClasses,\
'one-versus-all', 'normalization', NumFeatures,\
SVMHandle)
read_samples_class_svm (SVMHandle, 'samples.mtf')
train_class_svm (SVMHandle, 0.001, 'default')
write_class_svm (SVMHandle, 'classifier.svm')
Result
If the parameters are valid the operator train_class_svm
returns the value 2 (
H_MSG_TRUE)
. If necessary, an exception is
raised.
Possible Predecessors
add_sample_class_svm,
read_samples_class_svm
Possible Successors
classify_class_svm,
write_class_svm,
create_class_lut_svm
Alternatives
train_dl_classifier_batch,
read_class_svm
See also
References
John Shawe-Taylor, Nello Cristianini: “Kernel Methods for Pattern
Analysis”; Cambridge University Press, Cambridge; 2004.
Bernhard Schölkopf, Alexander J.Smola: “Learning with Kernels”;
MIT Press, London; 1999.
Module
Foundation