Builder

NetworkDefinitionCreationFlag

tensorrt.NetworkDefinitionCreationFlag

List of immutable network properties expressed at network creation time. For example, to enable explicit batch mode, pass a value of 1 << NetworkDefinitionCreationFlag.EXPLICIT_BATCH to create_network()

Members:

EXPLICIT_BATCH : Specify that the network should be created with an explicit batch dimension.

EXPLICIT_PRECISION : Specify that the network contains explicit quantization and dequantization scale layers.

Builder

class tensorrt.Builder(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None

Builds an ICudaEngine from a INetworkDefinition .

Variables

  • max_batch_size – int The maximum batch size which can be used at execution time, and also the batch size for which the ICudaEngine will be optimized.

  • platform_has_tf32 – bool Whether the platform has tf32 support.

  • platform_has_fast_fp16 – bool Whether the platform has fast native fp16.

  • platform_has_fast_int8 – bool Whether the platform has fast native int8.

  • max_DLA_batch_size – int The maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

  • num_DLA_cores – int The number of DLA engines available to this builder.

  • error_recorder – IErrorRecorder Application-implemented error reporting interface for TensorRT objects.

  • gpu_allocator – IGpuAllocator The GPU allocator to be used by the Builder . All GPU memory acquired will use this allocator. If set to None, the default allocator will be used.

  • logger – ILogger The logger provided when creating the refitter.

Parameters

logger – The logger to use.

__del__(self: tensorrt.tensorrt.Builder) → None
__exit__(exc_type, exc_value, traceback)

Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.

__init__(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None
Parameters

logger – The logger to use.

build_engine(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.ICudaEngine

Builds an engine for the given INetworkDefinition and IBuilderConfig .

This enables the builder to build multiple engines based on the same network definition, but with different builder configurations.

Parameters
Returns

A new ICudaEngine .

build_serialized_network(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.IHostMemory

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig .

This function allows building and serialization of a network without creating an engine.

Parameters

  • network – Network definition.

  • config – Builder configuration.

Returns

A pointer to a IHostMemory object that contains a serialized network.

create_builder_config(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IBuilderConfig

Create a builder configuration object.

See IBuilderConfig

create_network(self: tensorrt.tensorrt.Builder, flags: int = 0) → tensorrt.tensorrt.INetworkDefinition

Create a INetworkDefinition object.

Parameters

flags – NetworkDefinitionCreationFlag s combined using bitwise OR. Default value is 0. This mimics the behavior of create_network() in TensorRT 5.1.

Returns

An empty TensorRT INetworkDefinition .

create_optimization_profile(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IOptimizationProfile

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to IOptimizationProfile.set_shape() must be made. Likewise, if there are any shape input tensors, the appropriate calls to IOptimizationProfile.set_shape_input() are required.

See IOptimizationProfile

is_network_supported(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → bool

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters

  • network – The network definition to check for configuration compliance.

  • config – The configuration of the builder to use when checking the network.

Given an INetworkDefinition and an IBuilderConfig , check if the network falls within the constraints of the builder configuration based on the EngineCapability , BuilderFlag , and DeviceType .

Returns

True if network is within the scope of the restrictions specified by the builder config, False otherwise. This function reports the conditions that are violated to the registered ErrorRecorder .

NOTE: This function will synchronize the cuda stream returned by config.profile_stream before returning.

reset(self: tensorrt.tensorrt.Builder) → None

Resets the builder state to default values.