IBuilderConfig

tensorrt.BuilderFlag

Enable FP16 layer selection

INT8 : Enable Int8 layer selection

DEBUG : Enable debugging of layers via synchronizing after every layer

GPU_FALLBACK : Enable layers marked to execute on GPU if layer cannot execute on DLA

STRICT_TYPES : Enables strict type constraints

REFIT : Enable building a refittable engine

Type:Members
Type:FP16
class tensorrt.IBuilderConfig
Variables:
  • min_timing_iterations – int The number of minimization iterations used when timing layers. When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in minimization.
  • avg_timing_iterations – int The number of averaging iterations used when timing layers. When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in averaging.
  • int8_calibrator – IInt8Calibrator Int8 Calibration interface. The calibrator is to minimize the information loss during the INT8 quantization process.
  • max_workspace_size – int The maximum workspace size. The maximum GPU temporary memory which the engine can use at execution time.
  • flags – int The build mode flags to turn on builder options for this network. The flags are listed in the BuilderFlags enum. The flags set configuration options to build the network. This should be in integer consisting of one or more BuilderFlag s, combined via binary OR. For example, 1 << BuilderFlag.FP16 | 1 << BuilderFlag.DEBUG.
  • profile_stream – int The handle for the CUDA stream that is used to profile this network.
  • num_optimization_profiles – int The number of optimization profiles.
  • default_device_type – tensorrt.DeviceType The default DeviceType to be used by the Builder.
  • DLA_core – int The DLA core that the engine executes on. Must be between 0 and N-1 where N is the number of available DLA cores.
__del__(self: tensorrt.tensorrt.IBuilderConfig) → None
__exit__(exc_type, exc_value, traceback)

Destroy this object, freeing all memory associated with it. This should be called to ensure that the object is cleaned up properly. Equivalent to invoking __del__()

__init__

Initialize self. See help(type(self)) for accurate signature.

add_optimization_profile(self: tensorrt.tensorrt.IBuilderConfig, profile: tensorrt.tensorrt.IOptimizationProfile) → int

Add an optimization profile.

This function must be called at least once if the network has dynamic or shape input tensors.

Parameters:profile – The new optimization profile, which must satisfy bool(profile) == True
Returns:The index of the optimization profile (starting from 0) if the input is valid, or -1 if the input is not valid.
can_run_on_DLA(self: tensorrt.tensorrt.IBuilderConfig, layer: tensorrt.tensorrt.ILayer) → bool

Check if the layer can run on DLA.

Parameters:layer – The layer to check
Returns:A bool indicating whether the layer can run on DLA
clear_flag(self: tensorrt.tensorrt.IBuilderConfig, flag: tensorrt.tensorrt.BuilderFlag) → None

Clear a single build mode flag.

Parameters:flag – The flag to clear.
get_device_type(self: tensorrt.tensorrt.IBuilderConfig, layer: tensorrt.tensorrt.ILayer) → tensorrt.tensorrt.DeviceType

Get the device that the layer executes on.

Parameters:layer – The layer to get the DeviceType for
Returns:The DeviceType of the layer
get_flag(self: tensorrt.tensorrt.IBuilderConfig, flag: tensorrt.tensorrt.BuilderFlag) → bool

Check if a build mode flag is set.

Parameters:flag – The flag to check.
Returns:A bool indicating whether the flag is set.
is_device_type_set(self: tensorrt.tensorrt.IBuilderConfig, layer: tensorrt.tensorrt.ILayer) → bool

Check if the DeviceType for a layer is explicitly set.

Parameters:layer – The layer to check for DeviceType
Returns:True if DeviceType is not default, False otherwise
reset(self: tensorrt.tensorrt.IBuilderConfig) → None

Resets the builder configuration to defaults. When initializing a builder config object, we can call this function.

reset_device_type(self: tensorrt.tensorrt.IBuilderConfig, layer: tensorrt.tensorrt.ILayer) → None

Reset the DeviceType for the given layer.

Parameters:layer – The layer to reset the DeviceType for
set_device_type(self: tensorrt.tensorrt.IBuilderConfig, layer: tensorrt.tensorrt.ILayer, device_type: tensorrt.tensorrt.DeviceType) → None

Set the device that this layer must execute on. If DeviceType is not set or is reset, TensorRT will use the default DeviceType set in the builder.

The DeviceType for a layer must be compatible with the safety flow (if specified). For example a layer cannot be marked for DLA execution while the builder is configured for kSAFE_GPU.

Parameters:
  • layer – The layer to set the DeviceType of
  • device_type – The DeviceType the layer must execute on
set_flag(self: tensorrt.tensorrt.IBuilderConfig, flag: tensorrt.tensorrt.BuilderFlag) → None

Set a single build mode flag.

Parameters:flag – The flag to set.