Skip to content

ONNX Runtime AI²

The ONNX Runtime AI² interface integrates the ONNX Runtime inference engine into HALCON. It provides a common interface for executing supported HALCON deep learning applications on various hardware platforms.

Interface Name ONNX Runtime AI²
Version 26.11.1.0
Release Date 2026-11-18
Available Architectures x64-win64 x64-linux aarch64-linux

Installation

You can install the ONNX Runtime AI² via MVTec Software Manager (SOM).

For information on installing and managing HALCON packages, refer to the HALCON Setup and Installation Guide.

The SOM places all required components in the appropriate location, making the interface immediately available within HALCON.

Additional software requirements may apply to individual execution providers.

Architecture

A HALCON application works with HALCON deep learning models and deep learning operators. The ONNX Runtime AI² interface connects these to ONNX Runtime, which uses an execution provider to execute supported model operations on the target hardware.

This separates the HALCON application from the underlying inference backend and hardware-specific APIs.

The general execution flow is:

graph TD
    id1["HALCON deep learning<br/>application"]
    id2["ONNX Runtime AI²"]
    id3["Execution Provider"]
    id4["Target hardware"]

    id1 --> id2 --> id3 --> id4

    classDef rounded rx:12,ry:12
    class id1,id2,id3,id4 rounded

Execution providers

Execution providers are the hardware-specific backends used by ONNX Runtime to execute deep learning models. ONNX Runtime provides a common inference interface, while execution providers connect this interface to specific hardware and inference technologies.

Execution provider Recommended use
CPU Execution Provider General-purpose inference on the CPU without requiring dedicated acceleration hardware.

The appropriate execution provider depends on the available hardware, model capabilities, precision, and performance requirements.

HALCON deep learning devices

HALCON applications can use query_available_dl_devices to query the deep-learning-capable devices available through the ONNX Runtime AI² interface. The returned DLDeviceHandle values identify devices that can subsequently be used with HALCON deep learning operators that accept a DLDeviceHandle.

The available devices depend on the target hardware, platform, installed execution providers, and supported configurations.

Device query

The operator query_available_dl_devices returns handles to deep-learning-capable devices available through the ONNX Runtime AI² interface.

For each returned device, every parameter specified in GenParamName must be equal to at least one of the corresponding values specified in GenParamValue. A parameter can have more than one value by specifying its name multiple times in GenParamName and providing different corresponding values in GenParamValue.

For the ONNX Runtime AI² interface, the following parameters are used to query the available devices:

Parameter Values Type Description
ai2 ['onnxruntime'] string Identifies the ONNX Runtime AI² interface providing the device.
execution_provider ['CPU'] string Identifies the ONNX Runtime execution provider used for inference.
type ['cpu', 'gpu', 'npu'] string Identifies the type of the device.
precisions ['float32', 'float16', 'bfloat16', 'int8'] string Specifies the inference precision supported by the device.

The available values and combinations depend on the execution providers supported and available on the target system.

Example

The following example queries the devices available through the ONNX Runtime AI² interface:

query_available_dl_devices ('ai2', 'onnxruntime', DLDeviceHandles)

The returned DLDeviceHandles contain handles to the available devices.

Model execution

A typical inference workflow consists of the following steps:

  1. A supported HALCON deep learning model is available for inference.
  2. A suitable DLDeviceHandle is selected or obtained using query_available_dl_devices.
  3. An optimized HALCON deep learning model is created for inference on the target device using optimize_dl_model_for_inference.
  4. HALCON deep learning operators execute the model on the selected device using the corresponding execution provider.

The inference result is returned to the HALCON application.

Usage of third-party libraries

This plugin depends on the ONNX Runtime as well as on other third-party libraries. See the full list including the copyright and license information in the thidparty_onnxruntime_ai2.txt file distributed together with the plugin package.

Release notes

Version 26.11.1.0 (2026-11-18)

  • Initial release of the ONNX Runtime AI² interface.
onnxruntime 26.11.1.0 Doc generated: 2026-11-17T07:40:23