Builder

NetworkDefinitionCreationFlag

tensorrt.NetworkDefinitionCreationFlag

List of immutable network properties expressed at network creation time. For example, to enable explicit batch mode, pass a value of 1 << int(NetworkDefinitionCreationFlag.STRONGLY_TYPED) to create_network()

Members:

EXPLICIT_BATCH : [DEPRECATED] Ignored because networks are always “explicit batch” in TensorRT 10.0.

STRONGLY_TYPED : Specify that every tensor in the network has a data type defined in the network following only type inference rules and the inputs/operator annotations. Setting layer precision and layer output types is not allowed, and the network output types will be inferred based on the input types and the type inference rules

Builder

class tensorrt.Builder(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) None

Builds an ICudaEngine from a INetworkDefinition .

Variables
  • platform_has_tf32bool Whether the platform has tf32 support.

  • platform_has_fast_fp16bool Whether the platform has fast native fp16.

  • platform_has_fast_int8bool Whether the platform has fast native int8.

  • max_DLA_batch_sizeint The maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

  • num_DLA_coresint The number of DLA engines available to this builder.

  • error_recorderIErrorRecorder Application-implemented error reporting interface for TensorRT objects.

  • gpu_allocatorIGpuAllocator The GPU allocator to be used by the Builder . All GPU memory acquired will use this allocator. If set to None, the default allocator will be used.

  • loggerILogger The logger provided when creating the refitter.

  • max_threadsint The maximum thread that can be used by the Builder.

Parameters

logger – The logger to use.

__del__(self: tensorrt.tensorrt.Builder) None
__exit__(exc_type, exc_value, traceback)

Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.

__init__(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) None
Parameters

logger – The logger to use.

build_serialized_network(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) tensorrt.tensorrt.IHostMemory

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig .

This function allows building and serialization of a network without creating an engine.

Parameters
  • network – Network definition.

  • config – Builder configuration.

Returns

A pointer to a IHostMemory object that contains a serialized network.

create_builder_config(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IBuilderConfig

Create a builder configuration object.

See IBuilderConfig

create_network(self: tensorrt.tensorrt.Builder, flags: int = 0) tensorrt.tensorrt.INetworkDefinition

Create a INetworkDefinition object.

Parameters

flagsNetworkDefinitionCreationFlag s combined using bitwise OR.

Returns

An empty TensorRT INetworkDefinition .

create_optimization_profile(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IOptimizationProfile

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to IOptimizationProfile.set_shape() must be made. Likewise, if there are any shape input tensors, the appropriate calls to IOptimizationProfile.set_shape_input() are required.

See IOptimizationProfile

get_plugin_registry(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IPluginRegistry

Get the local plugin registry that can be used by the builder.

Returns

The local plugin registry that can be used by the builder.

is_network_supported(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) bool

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters
  • network – The network definition to check for configuration compliance.

  • config – The configuration of the builder to use when checking the network.

Given an INetworkDefinition and an IBuilderConfig , check if the network falls within the constraints of the builder configuration based on the EngineCapability , BuilderFlag , and DeviceType .

Returns

True if network is within the scope of the restrictions specified by the builder config, False otherwise. This function reports the conditions that are violated to the registered ErrorRecorder .

NOTE: This function will synchronize the cuda stream returned by config.profile_stream before returning.

reset(self: tensorrt.tensorrt.Builder) None

Resets the builder state to default values.