Builder¶
NetworkDefinitionCreationFlag¶
-
tensorrt.
NetworkDefinitionCreationFlag
¶ List of immutable network properties expressed at network creation time. For example, to enable explicit batch mode, pass a value of
1 << NetworkDefinitionCreationFlag.EXPLICIT_BATCH
tocreate_network()
Members:
EXPLICIT_BATCH : Specify that the network should be created with an explicit batch dimension.
EXPLICIT_PRECISION : Specify that the network contains explicit quantization and dequantization scale layers.
Builder¶
-
class
tensorrt.
Builder
(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None¶ Builds an
ICudaEngine
from aINetworkDefinition
.- Variables
max_batch_size –
int
The maximum batch size which can be used at execution time, and also the batch size for which theICudaEngine
will be optimized.max_workspace_size –
int
The maximum GPU temporary memory which theICudaEngine
can use at execution time.debug_sync –
bool
Whether theBuilder
should use debug synchronization. If this is true, theBuilder
will synchronize after timing each layer, and report the layer name. It can be useful when diagnosing issues at build time.min_find_iterations –
int
The number of minimization iterations used when timing layers. When timing layers, theBuilder
minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in minimization.average_find_iterations –
int
The number of averaging iterations used when timing layers. When timing layers, theBuilder
minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in averaging.platform_has_tf32 –
bool
Whether the platform has tf32 support.platform_has_fast_fp16 –
bool
Whether the platform has fast native fp16.platform_has_fast_int8 –
bool
Whether the platform has fast native int8.int8_mode –
bool
Whether Int8 mode is used.int8_calibrator –
IInt8Calibrator
The Int8 Calibration interface.fp16_mode –
bool
Whether or not 16-bit kernels are permitted. DuringICudaEngine
build fp16 kernels will also be tried when this mode is enabled.strict_type_constraints –
bool
When strict type constraints is set, TensorRT will choose the type constraints that conforms to type constraints. If the flag is not enabled higher precision implementation may be chosen if it results in higher performance.refittable –
bool
Whether anICudaEngine
will be refittable.error_recorder –
IErrorRecorder
Reference counted application-implemented error reporting interface for TensorRT objects.
- Parameters
logger – The logger to use.
-
__del__
(self: tensorrt.tensorrt.Builder) → None¶
-
__exit__
(exc_type, exc_value, traceback)¶ Destroy this object, freeing all memory associated with it. This should be called to ensure that the object is cleaned up properly. Equivalent to invoking
__del__()
-
__init__
(self: tensorrt.tensorrt.Builder, logger: tensorrt.tensorrt.ILogger) → None¶ - Parameters
logger – The logger to use.
-
build_cuda_engine
(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition) → tensorrt.tensorrt.ICudaEngine¶ Builds an
ICudaEngine
from aINetworkDefinition
.- Parameters
network – The TensorRT
INetworkDefinition
.- Returns
A new
ICudaEngine
.
-
build_engine
(self: tensorrt.tensorrt.Builder, network: tensorrt.tensorrt.INetworkDefinition, config: tensorrt.tensorrt.IBuilderConfig) → tensorrt.tensorrt.ICudaEngine¶ Builds an engine for the given
INetworkDefinition
andIBuilderConfig
.This enables the builder to build multiple engines based on the same network definition, but with different builder configurations.
- Parameters
network – The TensorRT
INetworkDefinition
.config – The TensorRT
IBuilderConfig
.
- Returns
A new
ICudaEngine
.
-
create_builder_config
(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IBuilderConfig¶ Create a builder configuration object.
See
IBuilderConfig
-
create_network
(self: tensorrt.tensorrt.Builder, flags: int = 0) → tensorrt.tensorrt.INetworkDefinition¶ Create a
INetworkDefinition
object.- Parameters
flags –
NetworkDefinitionCreationFlag
s combined using bitwise OR. Default value is 0. This mimics the behavior of create_network() in TensorRT 5.1.- Returns
An empty TensorRT
INetworkDefinition
.
-
create_optimization_profile
(self: tensorrt.tensorrt.Builder) → tensorrt.tensorrt.IOptimizationProfile¶ Create a new optimization profile.
If the network has any dynamic input tensors, the appropriate calls to
IOptimizationProfile.set_shape()
must be made. Likewise, if there are any shape input tensors, the appropriate calls toIOptimizationProfile.set_shape_input()
are required.