Builder
NetworkDefinitionCreationFlag
- tensorrt.NetworkDefinitionCreationFlag
List of immutable network properties expressed at network creation time. For example, to enable explicit batch mode, pass a value of
1 << int(NetworkDefinitionCreationFlag.STRONGLY_TYPED)
tocreate_network()
Members:
EXPLICIT_BATCH : [DEPRECATED] Ignored because networks are always “explicit batch” in TensorRT 10.0.
STRONGLY_TYPED : Specify that every tensor in the network has a data type defined in the network following only type inference rules and the inputs/operator annotations. Setting layer precision and layer output types is not allowed, and the network output types will be inferred based on the input types and the type inference rules
Builder
- class tensorrt.Builder(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) None
Builds an
ICudaEngine
from aINetworkDefinition
.- Variables
platform_has_tf32 –
bool
Whether the platform has tf32 support.platform_has_fast_fp16 –
bool
Whether the platform has fast native fp16.platform_has_fast_int8 –
bool
Whether the platform has fast native int8.max_DLA_batch_size –
int
The maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.num_DLA_cores –
int
The number of DLA engines available to this builder.error_recorder –
IErrorRecorder
Application-implemented error reporting interface for TensorRT objects.gpu_allocator –
IGpuAllocator
The GPU allocator to be used by theBuilder
. All GPU memory acquired will use this allocator. If set toNone
, the default allocator will be used.logger –
ILogger
The logger provided when creating the refitter.max_threads –
int
The maximum thread that can be used by theBuilder
.
- Parameters
logger – The logger to use.
- __del__(self: tensorrt.tensorrt.Builder) None
- __exit__(exc_type, exc_value, traceback)
Context managers are deprecated and have no effect. Objects are automatically freed when the reference count reaches 0.
- __init__(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) None
- Parameters
logger – The logger to use.
- build_serialized_network(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) tensorrt.tensorrt.IHostMemory
Builds and serializes a network for the given
INetworkDefinition
andIBuilderConfig
.This function allows building and serialization of a network without creating an engine.
- Parameters
network – Network definition.
config – Builder configuration.
- Returns
A pointer to a
IHostMemory
object that contains a serialized network.
- create_builder_config(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IBuilderConfig
Create a builder configuration object.
See
IBuilderConfig
- create_network(self: tensorrt.tensorrt.Builder, flags: int = 0) tensorrt.tensorrt.INetworkDefinition
Create a
INetworkDefinition
object.- Parameters
flags –
NetworkDefinitionCreationFlag
s combined using bitwise OR.- Returns
An empty TensorRT
INetworkDefinition
.
- create_optimization_profile(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IOptimizationProfile
Create a new optimization profile.
If the network has any dynamic input tensors, the appropriate calls to
IOptimizationProfile.set_shape()
must be made. Likewise, if there are any shape input tensors, the appropriate calls toIOptimizationProfile.set_shape_input()
are required.
- get_plugin_registry(self: tensorrt.tensorrt.Builder) tensorrt.tensorrt.IPluginRegistry
Get the local plugin registry that can be used by the builder.
- Returns
The local plugin registry that can be used by the builder.
- is_network_supported(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) bool
Checks that a network is within the scope of the
IBuilderConfig
settings.- Parameters
network – The network definition to check for configuration compliance.
config – The configuration of the builder to use when checking the network.
Given an
INetworkDefinition
and anIBuilderConfig
, check if the network falls within the constraints of the builder configuration based on theEngineCapability
,BuilderFlag
, andDeviceType
.- Returns
True
if network is within the scope of the restrictions specified by the builder config,False
otherwise. This function reports the conditions that are violated to the registeredErrorRecorder
.
NOTE: This function will synchronize the cuda stream returned by
config.profile_stream
before returning.
- reset(self: tensorrt.tensorrt.Builder) None
Resets the builder state to default values.