TensorRT 8.5.3
nvinfer1::IRefitter Class Reference

Updates weights in an engine. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRefitter:
nvinfer1::INoCopy

Public Member Functions

virtual ~IRefitter () noexcept=default
 
bool setWeights (char const *layerName, WeightsRole role, Weights weights) noexcept
 Specify new weights for a layer of given name. Returns true on success, or false if new weights are rejected. Possible reasons for rejection are: More...
 
bool refitCudaEngine () noexcept
 Updates associated engine. Return true if successful. More...
 
int32_t getMissing (int32_t size, char const **layerNames, WeightsRole *roles) noexcept
 Get description of missing weights. More...
 
int32_t getAll (int32_t size, char const **layerNames, WeightsRole *roles) noexcept
 Get description of all weights that could be refit. More...
 
TRT_DEPRECATED void destroy () noexcept
 
bool setDynamicRange (char const *tensorName, float min, float max) noexcept
 
float getDynamicRangeMin (char const *tensorName) const noexcept
 Get minimum of dynamic range. More...
 
float getDynamicRangeMax (char const *tensorName) const noexcept
 Get maximum of dynamic range. More...
 
int32_t getTensorsWithDynamicRange (int32_t size, char const **tensorNames) const noexcept
 Get names of all tensors that have refittable dynamic ranges. More...
 
void setErrorRecorder (IErrorRecorder *recorder) noexcept
 Set the ErrorRecorder for this interface. More...
 
IErrorRecordergetErrorRecorder () const noexcept
 Get the ErrorRecorder assigned to this interface. More...
 
bool setNamedWeights (char const *name, Weights weights) noexcept
 Specify new weights of given name. More...
 
int32_t getMissingWeights (int32_t size, char const **weightsNames) noexcept
 Get names of missing weights. More...
 
int32_t getAllWeights (int32_t size, char const **weightsNames) noexcept
 Get names of all weights that could be refit. More...
 
ILoggergetLogger () const noexcept
 get the logger with which the refitter was created More...
 
bool setMaxThreads (int32_t maxThreads) noexcept
 Set the maximum number of threads. More...
 
int32_t getMaxThreads () const noexcept
 get the maximum number of threads that can be used by the refitter. More...
 

Protected Attributes

apiv::VRefitter * mImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

Updates weights in an engine.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRefitter()

virtual nvinfer1::IRefitter::~IRefitter ( )
virtualdefaultnoexcept

Member Function Documentation

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRefitter::destroy ( )
inlinenoexcept
Deprecated:
Deprecated in TRT 8.0. Superseded by delete.
Warning
Calling destroy on a managed pointer will result in a double-free error.

◆ getAll()

int32_t nvinfer1::IRefitter::getAll ( int32_t  size,
char const **  layerNames,
WeightsRole roles 
)
inlinenoexcept

Get description of all weights that could be refit.

Parameters
sizeThe number of items that can be safely written to a non-null layerNames or roles.
layerNamesWhere to write the layer names.
rolesWhere to write the weights roles.
Returns
The number of Weights that could be refit.

If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.

◆ getAllWeights()

int32_t nvinfer1::IRefitter::getAllWeights ( int32_t  size,
char const **  weightsNames 
)
inlinenoexcept

Get names of all weights that could be refit.

Parameters
sizeThe number of weights names that can be safely written to.
weightsNamesThe names of the weights to be updated, or nullptr for unnamed weights.
Returns
The number of Weights that could be refit.

If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.

◆ getDynamicRangeMax()

float nvinfer1::IRefitter::getDynamicRangeMax ( char const *  tensorName) const
inlinenoexcept

Get maximum of dynamic range.

Returns
Maximum of dynamic range.

If the dynamic range was never set, returns the maximum computed during calibration.

Warning
The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.

◆ getDynamicRangeMin()

float nvinfer1::IRefitter::getDynamicRangeMin ( char const *  tensorName) const
inlinenoexcept

Get minimum of dynamic range.

Returns
Minimum of dynamic range.

If the dynamic range was never set, returns the minimum computed during calibration.

Warning
The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IRefitter::getErrorRecorder ( ) const
inlinenoexcept

Get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IRefitter::getLogger ( ) const
inlinenoexcept

get the logger with which the refitter was created

Returns
the logger

◆ getMaxThreads()

int32_t nvinfer1::IRefitter::getMaxThreads ( ) const
inlinenoexcept

get the maximum number of threads that can be used by the refitter.

Retrieves the maximum number of threads that can be used by the refitter.

Returns
The maximum number of threads that can be used by the refitter.
See also
setMaxThreads()

◆ getMissing()

int32_t nvinfer1::IRefitter::getMissing ( int32_t  size,
char const **  layerNames,
WeightsRole roles 
)
inlinenoexcept

Get description of missing weights.

For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.

Parameters
sizeThe number of items that can be safely written to a non-null layerNames or roles.
layerNamesWhere to write the layer names.
rolesWhere to write the weights roles.
Returns
The number of missing Weights.

If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.

◆ getMissingWeights()

int32_t nvinfer1::IRefitter::getMissingWeights ( int32_t  size,
char const **  weightsNames 
)
inlinenoexcept

Get names of missing weights.

For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.

Parameters
sizeThe number of weights names that can be safely written to.
weightsNamesThe names of the weights to be updated, or nullptr for unnamed weights.
Returns
The number of missing Weights.

If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.

◆ getTensorsWithDynamicRange()

int32_t nvinfer1::IRefitter::getTensorsWithDynamicRange ( int32_t  size,
char const **  tensorNames 
) const
inlinenoexcept

Get names of all tensors that have refittable dynamic ranges.

Parameters
sizeThe number of items that can be safely written to a non-null tensorNames.
tensorNamesWhere to write the layer names.
Returns
The number of Weights that could be refit.

If tensorNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.

◆ refitCudaEngine()

bool nvinfer1::IRefitter::refitCudaEngine ( )
inlinenoexcept

Updates associated engine. Return true if successful.

Failure occurs if getMissing() != 0 before the call.

The behavior is undefined if the engine has pending enqueued work.

Extant IExecutionContexts associated with the engine should not be used afterwards. Instead, create new IExecutionContexts after refitting.

◆ setDynamicRange()

bool nvinfer1::IRefitter::setDynamicRange ( char const *  tensorName,
float  min,
float  max 
)
inlinenoexcept

Update dynamic range for a tensor.

Parameters
tensorNameThe name of an ITensor in the network.
minThe minimum of the dynamic range for the tensor.
maxThe maximum of the dynamic range for the tensor.
Returns
True if successful; false otherwise.

Returns false if there is no Int8 engine tensor derived from a network tensor of that name. If successful, then getMissing may report that some weights need to be supplied.

Warning
The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.

◆ setErrorRecorder()

void nvinfer1::IRefitter::setErrorRecorder ( IErrorRecorder recorder)
inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()

◆ setMaxThreads()

bool nvinfer1::IRefitter::setMaxThreads ( int32_t  maxThreads)
inlinenoexcept

Set the maximum number of threads.

Parameters
maxThreadsThe maximum number of threads that can be used by the refitter.
Returns
True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

◆ setNamedWeights()

bool nvinfer1::IRefitter::setNamedWeights ( char const *  name,
Weights  weights 
)
inlinenoexcept

Specify new weights of given name.

Parameters
nameThe name of the weights to be refit.
weightsThe new weights to associate with the name.

Returns true on success, or false if new weights are rejected. Possible reasons for rejection are:

  • The name of weights is nullptr or does not correspond to any refittable weights.
  • The number of weights is inconsistent with the original specification.

Modifying the weights before method refitCudaEngine() completes will result in undefined behavior.

Warning
The string name must be null-terminated, and be at most 4096 bytes including the terminator.

◆ setWeights()

bool nvinfer1::IRefitter::setWeights ( char const *  layerName,
WeightsRole  role,
Weights  weights 
)
inlinenoexcept

Specify new weights for a layer of given name. Returns true on success, or false if new weights are rejected. Possible reasons for rejection are:

  • There is no such layer by that name.
  • The layer does not have weights with the specified role.
  • The number of weights is inconsistent with the layer’s original specification.

Modifying the weights before method refit() completes will result in undefined behavior.

Warning
The string layerName must be null-terminated, and be at most 4096 bytes including the terminator.

Member Data Documentation

◆ mImpl

apiv::VRefitter* nvinfer1::IRefitter::mImpl
protected

The documentation for this class was generated from the following file: