Builder

NetworkDefinitionCreationFlag

tensorrt.NetworkDefinitionCreationFlag

List of immutable network properties expressed at network creation time. For example, to enable strongly typed mode, pass a value of 1 << int(NetworkDefinitionCreationFlag.STRONGLY_TYPED) to create_network()

Members:

EXPLICIT_BATCH : [DEPRECATED] Ignored because networks are always “explicit batch” in TensorRT 10.0.

STRONGLY_TYPED : Specify that every tensor in the network has a data type defined in the network following only type inference rules and the inputs/operator annotations. Setting layer precision and layer output types is not allowed, and the network output types will be inferred based on the input types and the type inference rules

PREFER_AOT_PYTHON_PLUGINS : If set, for a Python plugin with both AOT and JIT implementations, the AOT implementation will be used.

PREFER_JIT_PYTHON_PLUGINS : If set, for a Python plugin with both AOT and JIT implementations, the JIT implementation will be used.

Builder

class tensorrt.Builder(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger)

Builds an ICudaEngine from a INetworkDefinition .

Variables:

  • platform_has_tf32 – bool Whether the platform has tf32 support.

  • platform_has_fast_fp16 – bool Whether the platform has fast native fp16.

  • platform_has_fast_int8 – bool Whether the platform has fast native int8.

  • max_DLA_batch_size – int The maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

  • num_DLA_cores – int The number of DLA engines available to this builder.

  • error_recorder – IErrorRecorder Application-implemented error reporting interface for TensorRT objects.

  • gpu_allocator – IGpuAllocator The GPU allocator to be used by the Builder . All GPU memory acquired will use this allocator. If set to None, the default allocator will be used.

  • logger – ILogger The logger provided when creating the refitter.

  • max_threads – int The maximum thread that can be used by the Builder.

Parameters:

logger – The logger to use.

__del__(self: tensorrt.tensorrt.Builder) → None
__exit__(exc_type, exc_value, traceback)

Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.

__init__(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None
Parameters:

logger – The logger to use.

build_engine_with_config(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.ICudaEngine

Builds a network for the given INetworkDefinition and IBuilderConfig .

This function allows building a network and creating an engine.

Parameters:

  • network – Network definition.

  • config – Builder configuration.

Returns:

A pointer to a ICudaEngine object that contains a built engine.

build_serialized_network(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.IHostMemory

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig .

This function allows building and serialization of a network without creating an engine.

Parameters:

  • network – Network definition.

  • config – Builder configuration.

Returns:

A pointer to a IHostMemory object that contains a serialized network.

build_serialized_network_to_stream(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig, writer: tensorrt.tensorrt.IStreamWriter) → bool

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig and write the serialized network to a writer stream.

This function allows building and serialization of a network without creating an engine.

Parameters:

  • network – Network definition.

  • config – Builder configuration.

  • writer – Output stream writer.

Returns:

True if build succeed, otherwise False.

create_builder_config(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IBuilderConfig

Create a builder configuration object.

See IBuilderConfig

create_network(self: tensorrt.tensorrt.Builder, flags: int = 0) → tensorrt.tensorrt.INetworkDefinition

Create a INetworkDefinition object.

Parameters:

flags – NetworkDefinitionCreationFlag s combined using bitwise OR.

Returns:

An empty TensorRT INetworkDefinition .

create_optimization_profile(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IOptimizationProfile

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to IOptimizationProfile.set_shape() must be made. Likewise, if there are any shape input tensors, the appropriate calls to IOptimizationProfile.set_shape_input() are required.

See IOptimizationProfile

get_plugin_registry(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IPluginRegistry

Get the local plugin registry that can be used by the builder.

Returns:

The local plugin registry that can be used by the builder.

is_network_supported(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → bool

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters:

  • network – The network definition to check for configuration compliance.

  • config – The configuration of the builder to use when checking the network.

Given an INetworkDefinition and an IBuilderConfig , check if the network falls within the constraints of the builder configuration based on the EngineCapability , BuilderFlag , and DeviceType .

Returns:

True if network is within the scope of the restrictions specified by the builder config, False otherwise. This function reports the conditions that are violated to the registered ErrorRecorder .

NOTE: This function will synchronize the cuda stream returned by config.profile_stream before returning.

reset(self: tensorrt.tensorrt.Builder) → None

Resets the builder state to default values.