Refitter

class tensorrt.Refitter(self: tensorrt.tensorrt.Refitter, engine: tensorrt.tensorrt.ICudaEngine, logger: tensorrt.tensorrt.ILogger) → None

Updates weights in an ICudaEngine .

Parameters

  • engine – The engine to refit.

  • logger – The logger to use.

__del__(self: tensorrt.tensorrt.Refitter) → None
__exit__(exc_type, exc_value, traceback)

Destroy this object, freeing all memory associated with it. This should be called to ensure that the object is cleaned up properly. Equivalent to invoking __del__()

__init__(self: tensorrt.tensorrt.Refitter, engine: tensorrt.tensorrt.ICudaEngine, logger: tensorrt.tensorrt.ILogger) → None
Parameters

  • engine – The engine to refit.

  • logger – The logger to use.

get_all(self: tensorrt.tensorrt.Refitter) → Tuple[List[str], List[tensorrt.tensorrt.WeightsRole]]

Get description of all weights that could be refit.

Returns

The names of layers with refittable weights, and the roles of those weights.

get_dynamic_range(self: tensorrt.tensorrt.Refitter, tensor_name: str) → tuple

Gets the dynamic range of a tensor. If the dynamic range was never set, returns the range computed during calibration.

Parameters

tensor_name – The name of the tensor whose dynamic range to retrieve.

Returns

Tuple[float, float] A tuple containing the [minimum, maximum] of the dynamic range.

get_missing(self: tensorrt.tensorrt.Refitter) → Tuple[List[str], List[tensorrt.tensorrt.WeightsRole]]

Get description of missing weights.

For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.

Returns

The names of layers with missing weights, and the roles of those weights.

get_tensors_with_dynamic_range(self: tensorrt.tensorrt.Refitter) → List[str]

Get names of all tensors that have refittable dynamic ranges.

Returns

The names of tensors with refittable dynamic ranges.

refit_cuda_engine(self: tensorrt.tensorrt.Refitter) → bool

Updates associated engine. Return True if successful.

Failure occurs if get_missing() != 0 before the call.

set_dynamic_range(self: tensorrt.tensorrt.Refitter, tensor_name: str, range: List[float]) → bool

Update dynamic range for a tensor.

Parameters

  • tensor_name – The name of the tensor whose dynamic range to update.

  • range – The new range.

Returns

True if successful, False otherwise.

Returns false if there is no Int8 engine tensor derived from a network tensor of that name. If successful, then get_missing() may report that some weights need to be supplied.

set_weights(self: tensorrt.tensorrt.Refitter, layer_name: str, role: tensorrt.tensorrt.WeightsRole, weights: tensorrt.tensorrt.Weights) → bool

Specify new weights for a layer of given name. Possible reasons for rejection are:

  • There is no such layer by that name.

  • The layer does not have weights with the specified role.

  • The number of weights is inconsistent with the layer’s original specification.

Modifying the weights before refit_cuda_engine() completes will result in undefined behavior.

Parameters

  • layer_name – The name of the layer.

  • role – The role of the weights. See WeightsRole for more information.

  • weights – The weights to refit with.

Returns

True on success, or False if new weights are rejected.