ONNX Parser Flags for DLA

Important

DLA is not supported in TensorRT 11.3.0 or in TensorRT 11.3.1 for DriveOS. TensorRT 10.7 was the last release that supported DLA. DLA was also not supported in TensorRT 11.0, 11.1, or 11.2.

The guidance in this section is reference material for supported earlier releases. Do not use it as a support claim for TensorRT 11.3.0 or for TensorRT 11.3.1 for DriveOS.

When building from an ONNX model, use OnnxParserFlag values with IParser::setFlags() to control DLA-specific parsing behavior. All three DLA-specific flags are disabled by default.

kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA

Enables UINT8 as a quantization data type and permits asymmetric quantization with nonzero zero-point values in QuantizeLinear and DequantizeLinear nodes. Without this flag, the parser converts UINT8 constants to INT32.

This flag is required for the asymmetric IQuantizeLayer and IDequantizeLayer support described in the DLA layer-restrictions pages. The scale must be FP32, and the zero-point must be UINT8. The target platform must be NVIDIA Orin.

kREPORT_CAPABILITY_DLA

Validates each parsed ONNX node against DLA support. If a resulting TensorRT layer cannot run on DLA, parsing fails with a descriptive error instead of allowing the layer to fall back to GPU.

Before calling IParser::parse(), provide a valid IBuilderConfig by calling IParser::setBuilderConfig(). When this flag is set, IParser::isSubGraphSupported() also reports capability in the DLA context.

kADJUST_FOR_DLA

Opportunistically rewrites or modifies layers during parsing to make them more amenable to DLA execution. For example, the parser can reshape tensors to 4D or adjust quantization formats.

The following C++ example enables all three flags for a DLA build:

parser->setFlags(
    (1U << static_cast<uint32_t>(
        OnnxParserFlag::kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA))
    | (1U << static_cast<uint32_t>(
        OnnxParserFlag::kREPORT_CAPABILITY_DLA))
    | (1U << static_cast<uint32_t>(
        OnnxParserFlag::kADJUST_FOR_DLA)));
parser->setBuilderConfig(builderConfig);

The builder configuration must remain valid until parsing is complete.