nvonnxparser Namespace Reference

The TensorRT ONNX parser API namespace. More...

Namespaces
namespace	anonymous_namespace{NvOnnxParser.h}

Classes
class	IOnnxConfig
	Configuration Manager Class. More...
class	IParser
	an object for parsing ONNX models into a TensorRT network definition More...
class	IParserError
	an object containing information about an error More...
class	IParserRefitter
	An interface designed to refit weights from an ONNX model. More...
class	IRefitterObserver
	Observer interface invoked by IParserRefitter once per refittable engine weight. More...
struct	RefitRecord
	One refittable-weight description emitted by IRefitterObserver. More...

Typedefs
using	OnnxParserFlags = uint32_t
	Represents one or more OnnxParserFlag values using binary OR operations, e.g., 1U << OnnxParserFlag::kNATIVE_INSTANCENORM. More...

Enumerations
enum class	ErrorCode : int {   kSUCCESS = 0 , kINTERNAL_ERROR = 1 , kMEM_ALLOC_FAILED = 2 , kMODEL_DESERIALIZE_FAILED = 3 ,   kINVALID_VALUE = 4 , kINVALID_GRAPH = 5 , kINVALID_NODE = 6 , kUNSUPPORTED_GRAPH = 7 ,   kUNSUPPORTED_NODE = 8 , kUNSUPPORTED_NODE_ATTR = 9 , kUNSUPPORTED_NODE_INPUT = 10 , kUNSUPPORTED_NODE_DATATYPE = 11 ,   kUNSUPPORTED_NODE_DYNAMIC = 12 , kUNSUPPORTED_NODE_SHAPE = 13 , kREFIT_FAILED = 14 }
	The type of error that the parser or refitter may return. More...
enum class	OnnxParserFlag : int32_t {   kNATIVE_INSTANCENORM = 0 , kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA = 1 , kREPORT_CAPABILITY_DLA = 2 , kENABLE_PLUGIN_OVERRIDE = 3 ,   kADJUST_FOR_DLA = 4 }
enum class	RefitTransformKind : int32_t {   kIDENTITY = 0 , kDOUBLE_TO_FLOAT = 1 , kBATCH_NORM_FOLD_SCALE = 2 , kBATCH_NORM_FOLD_BIAS = 3 ,   kCONSTANT_NODE = 4 , kCONSTANT_OF_SHAPE = 5 }
	Identifies how a refittable engine weight is produced from one or more ONNX initializers or node attributes. More...

Functions
TENSORRTAPI IOnnxConfig *	createONNXConfig ()
template<typename T >
constexpr int32_t	EnumMax () noexcept=delete
template<>
constexpr int32_t	EnumMax< ErrorCode > () noexcept
	Specialization. See nvonnxparser::EnumMax() for details. More...
template<>
constexpr int32_t	EnumMax< OnnxParserFlag > () noexcept
	Specialization. See nvonnxparser::EnumMax() for details. More...
template<>
constexpr int32_t	EnumMax< RefitTransformKind > () noexcept
	Specialization. See nvonnxparser::EnumMax() for details. More...

Detailed Description

The TensorRT ONNX parser API namespace.

Typedef Documentation

OnnxParserFlags

using nvonnxparser::OnnxParserFlags = typedef uint32_t

Represents one or more OnnxParserFlag values using binary OR operations, e.g., 1U << OnnxParserFlag::kNATIVE_INSTANCENORM.

See also

IParser::setFlags() and IParser::getFlags()

Enumeration Type Documentation

ErrorCode

enum class nvonnxparser::ErrorCode : int

strong

The type of error that the parser or refitter may return.

Enumerator
kSUCCESS
kINTERNAL_ERROR
kMEM_ALLOC_FAILED
kMODEL_DESERIALIZE_FAILED
kINVALID_VALUE
kINVALID_GRAPH
kINVALID_NODE
kUNSUPPORTED_GRAPH
kUNSUPPORTED_NODE
kUNSUPPORTED_NODE_ATTR
kUNSUPPORTED_NODE_INPUT
kUNSUPPORTED_NODE_DATATYPE
kUNSUPPORTED_NODE_DYNAMIC
kUNSUPPORTED_NODE_SHAPE
kREFIT_FAILED

OnnxParserFlag

enum class nvonnxparser::OnnxParserFlag : int32_t

strong

Enumerator
kNATIVE_INSTANCENORM	Parse the ONNX model into the INetworkDefinition with the intention of using TensorRT's native layer implementation over the plugin implementation for InstanceNormalization nodes. This flag is required when building version-compatible or hardware-compatible engines. This flag is set to be ON by default.
kENABLE_UINT8_AND_ASYMMETRIC_QUANTIZATION_DLA	Enable UINT8 as a quantization data type and asymmetric quantization with non-zero zero-point values in Quantize and Dequantize nodes. This flag is set to be OFF by default. The resulting engine must be built targeting DLA version >= 3.16.
kREPORT_CAPABILITY_DLA	Parse the ONNX model with per-node validation for DLA. If the model is not fully supported by DLA, then parsing will fail. If this flag is set, isSubGraphSupported() will also return capability in the context of DLA support. When this flag is set, a valid IBuilderConfig must be provided to the parser via setBuilderConfig().
kENABLE_PLUGIN_OVERRIDE	Allow a loaded plugin with the same name as an ONNX operator type to override the default ONNX implementation, even if the plugin namespace attribute is not set. Useful for custom plugins that replace standard ONNX operators, such as alternative implementations for better performance. This flag is set to be OFF by default.
kADJUST_FOR_DLA	Opportunistically rewrite or modify layers to make them more amenable to running on DLA.

RefitTransformKind

enum class nvonnxparser::RefitTransformKind : int32_t

strong

Identifies how a refittable engine weight is produced from one or more ONNX initializers or node attributes.

Emitted by IParserRefitter through IRefitterObserver to describe how each refittable engine weight is sourced. A consumer can record these descriptions at build time and replay them at engine-load time to refit directly via nvinfer1::IRefitter::setNamedWeights, without invoking the ONNX parser again.

Enumerator
kIDENTITY	Source is one ONNX initializer; refit data equals the initializer data verbatim.
kDOUBLE_TO_FLOAT	Source is one ONNX initializer of DOUBLE type; refit data is the FLOAT cast.
kBATCH_NORM_FOLD_SCALE	Source is the four scale/bias/mean/variance initializers of a BatchNormalization node. Refit data is combinedScale[i] = scale[i] / sqrt(variance[i] + epsilon).
kBATCH_NORM_FOLD_BIAS	Source is the four scale/bias/mean/variance initializers of a BatchNormalization node. Refit data is combinedBias[i] = bias[i] - mean[i] * combinedScale[i].
kCONSTANT_NODE	Source is the value attribute of a Constant node. The bytes are carried in RefitRecord::fixedData since they are not available as a separate ONNX initializer.
kCONSTANT_OF_SHAPE	Source is the value attribute of a ConstantOfShape node (defaulting to 0.0 if absent). The bytes are carried in RefitRecord::fixedData.

Function Documentation

createONNXConfig()

TENSORRTAPI IOnnxConfig * nvonnxparser::createONNXConfig

(

)

EnumMax()

template<typename T >

constexpr int32_t nvonnxparser::EnumMax ( )

constexprdeletenoexcept

Returns

the numerical value of the highest-valued enumerator for type T. It must be specialized for each enum type that uses it. {$nv-internal-release begin}

Note

This is different from nvinfer1::EnumMax<T>(), which is one more than that. {$nv-internal-release end}

EnumMax< ErrorCode >()

template<>

constexpr int32_t nvonnxparser::EnumMax< ErrorCode > ( )

constexprnoexcept

Specialization. See nvonnxparser::EnumMax() for details.

EnumMax< OnnxParserFlag >()

template<>

constexpr int32_t nvonnxparser::EnumMax< OnnxParserFlag > ( )

constexprnoexcept

Specialization. See nvonnxparser::EnumMax() for details.

EnumMax< RefitTransformKind >()

template<>

constexpr int32_t nvonnxparser::EnumMax< RefitTransformKind > ( )

constexprnoexcept

Specialization. See nvonnxparser::EnumMax() for details.