ONNX Parser Flags for DLA

Important

DLA is not supported for Linux SBSA, NVIDIA JetPack, or NVIDIA DriveOS deployments in TensorRT 11.4.0. Refer to Working with DLA for the platforms and packages that support DLA.

When building from an ONNX model, use OnnxParserFlag values with IParser::setFlags() to control DLA-specific parsing behavior. All three DLA-specific flags are disabled by default.

kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA

Enables UINT8 as a quantization data type and permits asymmetric quantization with nonzero zero-point values in QuantizeLinear and DequantizeLinear nodes. Without this flag, the parser converts UINT8 constants to INT32.

This flag is required for the asymmetric IQuantizeLayer and IDequantizeLayer support described in DLA Supported Layers for Windows on ARM. The scale must be FP32, and the zero-point must be UINT8. The target platform must be NVIDIA Orin or Windows on ARM.

kREPORT_CAPABILITY_DLA

Validates each parsed ONNX node against DLA support. If a resulting TensorRT layer cannot run on DLA, parsing fails with a descriptive error instead of allowing the layer to fall back to GPU.

Before calling IParser::parse(), provide a valid IBuilderConfig by calling IParser::setBuilderConfig(). When this flag is set, IParser::isSubGraphSupported() also reports capability in the DLA context.

kADJUST_FOR_DLA

Opportunistically rewrites or modifies layers during parsing to make them more amenable to DLA execution. For example, the parser can reshape tensors to 4D or adjust quantization formats.

The following C++ example enables all three flags for a DLA build:

parser->setFlags(
    (1U << static_cast<uint32_t>(
        OnnxParserFlag::kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA))
    | (1U << static_cast<uint32_t>(
        OnnxParserFlag::kREPORT_CAPABILITY_DLA))
    | (1U << static_cast<uint32_t>(
        OnnxParserFlag::kADJUST_FOR_DLA)));
parser->setBuilderConfig(builderConfig);

The builder configuration must remain valid until parsing is complete.