Builder

NetworkDefinitionCreationFlag

tensorrt.NetworkDefinitionCreationFlag

List of immutable network properties expressed at network creation time. For example, to enable explicit batch mode, pass a value of 1 << NetworkDefinitionCreationFlag.EXPLICIT_BATCH to create_network()

Members:

EXPLICIT_BATCH : Specify that the network should be created with an explicit batch dimension.

EXPLICIT_PRECISION : Specify that the network contains explicit quantization and dequantization scale layers.

Builder

class tensorrt.Builder(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None

Builds an ICudaEngine from a INetworkDefinition .

Variables
  • max_batch_sizeint The maximum batch size which can be used at execution time, and also the batch size for which the ICudaEngine will be optimized.

  • platform_has_tf32bool Whether the platform has tf32 support.

  • platform_has_fast_fp16bool Whether the platform has fast native fp16.

  • platform_has_fast_int8bool Whether the platform has fast native int8.

  • max_DLA_batch_sizeint The maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

  • num_DLA_coresint The number of DLA engines available to this builder.

  • error_recorderIErrorRecorder Application-implemented error reporting interface for TensorRT objects.

  • gpu_allocatorIGpuAllocator The GPU allocator to be used by the Builder . All GPU memory acquired will use this allocator. If set to None, the default allocator will be used.

Parameters

logger – The logger to use.

__del__(self: tensorrt.tensorrt.Builder) → None
__exit__(exc_type, exc_value, traceback)

Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.

__init__(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None
Parameters

logger – The logger to use.

build_engine(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.ICudaEngine

Builds an engine for the given INetworkDefinition and IBuilderConfig .

This enables the builder to build multiple engines based on the same network definition, but with different builder configurations.

Parameters
Returns

A new ICudaEngine .

build_serialized_network(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.IHostMemory

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig .

This function allows building and serialization of a network without creating an engine.

Parameters
  • network – Network definition.

  • config – Builder configuration.

Returns

A pointer to a IHostMemory object that contains a serialized network.

create_builder_config(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IBuilderConfig

Create a builder configuration object.

See IBuilderConfig

create_network(self: tensorrt.tensorrt.Builder, flags: int = 0) → tensorrt.tensorrt.INetworkDefinition

Create a INetworkDefinition object.

Parameters

flagsNetworkDefinitionCreationFlag s combined using bitwise OR. Default value is 0. This mimics the behavior of create_network() in TensorRT 5.1.

Returns

An empty TensorRT INetworkDefinition .

create_optimization_profile(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IOptimizationProfile

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to IOptimizationProfile.set_shape() must be made. Likewise, if there are any shape input tensors, the appropriate calls to IOptimizationProfile.set_shape_input() are required.

See IOptimizationProfile

is_network_supported(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → bool

Checks that a network is within the scope of the IBuilderConfig settings.

Parameters
  • network – The network definition to check for configuration compliance.

  • config – The configuration of the builder to use when checking the network.

Given an INetworkDefinition and an IBuilderConfig , check if the network falls within the constraints of the builder configuration based on the EngineCapability , BuilderFlag , and DeviceType .

Returns

True if the network is within the scope of the restrictions specified by the builder config, False otherwise. This function reports the conditions that are violated to the registered ErrorRecorder .

NOTE: This function will synchronize the cuda stream returned by config.profile_stream before returning.