Updates weights in an engine.
More...
#include <NvInferRuntime.h>
|
bool | setWeights (const char *layerName, WeightsRole role, Weights weights) noexcept |
| Specify new weights for a layer of given name. Returns true on success, or false if new weights are rejected. Possible reasons for rejection are: More...
|
|
bool | refitCudaEngine () noexcept |
| Updates associated engine. Return true if successful. More...
|
|
int32_t | getMissing (int32_t size, const char **layerNames, WeightsRole *roles) noexcept |
| Get description of missing weights. More...
|
|
int32_t | getAll (int32_t size, const char **layerNames, WeightsRole *roles) noexcept |
| Get description of all weights that could be refit. More...
|
|
TRT_DEPRECATED void | destroy () noexcept |
|
bool | setDynamicRange (const char *tensorName, float min, float max) noexcept |
|
float | getDynamicRangeMin (const char *tensorName) const noexcept |
| Get minimum of dynamic range. More...
|
|
float | getDynamicRangeMax (const char *tensorName) const noexcept |
| Get maximum of dynamic range. More...
|
|
int32_t | getTensorsWithDynamicRange (int32_t size, const char **tensorNames) const noexcept |
| Get names of all tensors that have refittable dynamic ranges. More...
|
|
void | setErrorRecorder (IErrorRecorder *recorder) noexcept |
| Set the ErrorRecorder for this interface. More...
|
|
IErrorRecorder * | getErrorRecorder () const noexcept |
| Get the ErrorRecorder assigned to this interface. More...
|
|
bool | setNamedWeights (const char *name, Weights weights) noexcept |
| Specify new weights of given name. More...
|
|
int32_t | getMissingWeights (int32_t size, const char **weightsNames) noexcept |
| Get names of missing weights. More...
|
|
int32_t | getAllWeights (int32_t size, const char **weightsNames) noexcept |
| Get names of all weights that could be refit. More...
|
|
ILogger * | getLogger () const noexcept |
| get the logger with which the refitter was created More...
|
|
Updates weights in an engine.
- Warning
- Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.
◆ destroy()
- Deprecated:
- Deprecated interface will be removed in TensorRT 10.0.
- Warning
- Calling destroy on a managed pointer will result in a double-free error.
◆ getAll()
int32_t nvinfer1::IRefitter::getAll |
( |
int32_t |
size, |
|
|
const char ** |
layerNames, |
|
|
WeightsRole * |
roles |
|
) |
| |
|
inlinenoexcept |
Get description of all weights that could be refit.
- Parameters
-
size | The number of items that can be safely written to a non-null layerNames or roles. |
layerNames | Where to write the layer names. |
roles | Where to write the weights roles. |
- Returns
- The number of Weights that could be refit.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
◆ getAllWeights()
int32_t nvinfer1::IRefitter::getAllWeights |
( |
int32_t |
size, |
|
|
const char ** |
weightsNames |
|
) |
| |
|
inlinenoexcept |
Get names of all weights that could be refit.
- Parameters
-
size | The number of weights names that can be safely written to. |
weightsNames | The names of the weights to be updated, or nullptr for unnamed weights. |
- Returns
- The number of Weights that could be refit.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
◆ getDynamicRangeMax()
float nvinfer1::IRefitter::getDynamicRangeMax |
( |
const char * |
tensorName | ) |
const |
|
inlinenoexcept |
Get maximum of dynamic range.
- Returns
- Maximum of dynamic range.
If the dynamic range was never set, returns the maximum computed during calibration.
◆ getDynamicRangeMin()
float nvinfer1::IRefitter::getDynamicRangeMin |
( |
const char * |
tensorName | ) |
const |
|
inlinenoexcept |
Get minimum of dynamic range.
- Returns
- Minimum of dynamic range.
If the dynamic range was never set, returns the minimum computed during calibration.
◆ getErrorRecorder()
Get the ErrorRecorder assigned to this interface.
Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.
- Returns
- A pointer to the IErrorRecorder object that has been registered.
- See also
- setErrorRecorder()
◆ getLogger()
ILogger * nvinfer1::IRefitter::getLogger |
( |
| ) |
const |
|
inlinenoexcept |
get the logger with which the refitter was created
- Returns
- the logger
◆ getMissing()
int32_t nvinfer1::IRefitter::getMissing |
( |
int32_t |
size, |
|
|
const char ** |
layerNames, |
|
|
WeightsRole * |
roles |
|
) |
| |
|
inlinenoexcept |
Get description of missing weights.
For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.
- Parameters
-
size | The number of items that can be safely written to a non-null layerNames or roles. |
layerNames | Where to write the layer names. |
roles | Where to write the weights roles. |
- Returns
- The number of missing Weights.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
◆ getMissingWeights()
int32_t nvinfer1::IRefitter::getMissingWeights |
( |
int32_t |
size, |
|
|
const char ** |
weightsNames |
|
) |
| |
|
inlinenoexcept |
Get names of missing weights.
For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.
- Parameters
-
size | The number of weights names that can be safely written to. |
weightsNames | The names of the weights to be updated, or nullptr for unnamed weights. |
- Returns
- The number of missing Weights.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
◆ getTensorsWithDynamicRange()
int32_t nvinfer1::IRefitter::getTensorsWithDynamicRange |
( |
int32_t |
size, |
|
|
const char ** |
tensorNames |
|
) |
| const |
|
inlinenoexcept |
Get names of all tensors that have refittable dynamic ranges.
- Parameters
-
size | The number of items that can be safely written to a non-null tensorNames. |
tensorNames | Where to write the layer names. |
- Returns
- The number of Weights that could be refit.
If tensorNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
◆ refitCudaEngine()
bool nvinfer1::IRefitter::refitCudaEngine |
( |
| ) |
|
|
inlinenoexcept |
Updates associated engine. Return true if successful.
Failure occurs if getMissing() != 0 before the call.
The behavior is undefined if the engine has pending enqueued work.
Extant IExecutionContexts associated with the engine should not be used afterwards. Instead, create new IExecutionContexts after refitting.
◆ setDynamicRange()
bool nvinfer1::IRefitter::setDynamicRange |
( |
const char * |
tensorName, |
|
|
float |
min, |
|
|
float |
max |
|
) |
| |
|
inlinenoexcept |
Update dynamic range for a tensor.
- Parameters
-
tensorName | The name of an ITensor in the network. |
min | The minimum of the dynamic range for the tensor. |
max | The maximum of the dynamic range for the tensor. |
- Returns
- True if successful; false otherwise.
Returns false if there is no Int8 engine tensor derived from a network tensor of that name. If successful, then getMissing may report that some weights need to be supplied.
◆ setErrorRecorder()
void nvinfer1::IRefitter::setErrorRecorder |
( |
IErrorRecorder * |
recorder | ) |
|
|
inlinenoexcept |
Set the ErrorRecorder for this interface.
Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.
If an error recorder is not set, messages will be sent to the global log stream.
- Parameters
-
recorder | The error recorder to register with this interface. |
- See also
- getErrorRecorder()
◆ setNamedWeights()
bool nvinfer1::IRefitter::setNamedWeights |
( |
const char * |
name, |
|
|
Weights |
weights |
|
) |
| |
|
inlinenoexcept |
Specify new weights of given name.
- Parameters
-
name | The name of the weights to be refit. |
weights | The new weights to associate with the name. |
Returns true on success, or false if new weights are rejected. Possible reasons for rejection are:
- The name of weights is nullptr or does not correspond to any refittable weights.
- The number of weights is inconsistent with the original specification.
Modifying the weights before method refitCudaEngine() completes will result in undefined behavior.
◆ setWeights()
bool nvinfer1::IRefitter::setWeights |
( |
const char * |
layerName, |
|
|
WeightsRole |
role, |
|
|
Weights |
weights |
|
) |
| |
|
inlinenoexcept |
Specify new weights for a layer of given name. Returns true on success, or false if new weights are rejected. Possible reasons for rejection are:
- There is no such layer by that name.
- The layer does not have weights with the specified role.
- The number of weights is inconsistent with the layer’s original specification.
Modifying the weights before method refit() completes will result in undefined behavior.
The documentation for this class was generated from the following file: