|
virtual | ~IRefitter () noexcept=default |
|
bool | setWeights (char const *layerName, WeightsRole role, Weights weights) noexcept |
| Specify new weights for a layer of given name. Returns true on success, or false if new weights are rejected. Possible reasons for rejection are: More...
|
|
bool | refitCudaEngine () noexcept |
| Refits associated engine. More...
|
|
int32_t | getMissing (int32_t size, char const **layerNames, WeightsRole *roles) noexcept |
| Get description of missing weights. More...
|
|
int32_t | getAll (int32_t size, char const **layerNames, WeightsRole *roles) noexcept |
| Get description of all weights that could be refit. More...
|
|
TRT_DEPRECATED bool | setDynamicRange (char const *tensorName, float min, float max) noexcept |
|
TRT_DEPRECATED float | getDynamicRangeMin (char const *tensorName) const noexcept |
| Get minimum of dynamic range. More...
|
|
TRT_DEPRECATED float | getDynamicRangeMax (char const *tensorName) const noexcept |
| Get maximum of dynamic range. More...
|
|
TRT_DEPRECATED int32_t | getTensorsWithDynamicRange (int32_t size, char const **tensorNames) const noexcept |
| Get names of all tensors that have refittable dynamic ranges. More...
|
|
void | setErrorRecorder (IErrorRecorder *recorder) noexcept |
| Set the ErrorRecorder for this interface. More...
|
|
IErrorRecorder * | getErrorRecorder () const noexcept |
| Get the ErrorRecorder assigned to this interface. More...
|
|
bool | setNamedWeights (char const *name, Weights weights) noexcept |
| Specify new weights of given name. More...
|
|
int32_t | getMissingWeights (int32_t size, char const **weightsNames) noexcept |
| Get names of missing weights. More...
|
|
int32_t | getAllWeights (int32_t size, char const **weightsNames) noexcept |
| Get names of all weights that could be refit. More...
|
|
ILogger * | getLogger () const noexcept |
| get the logger with which the refitter was created More...
|
|
bool | setMaxThreads (int32_t maxThreads) noexcept |
| Set the maximum number of threads. More...
|
|
int32_t | getMaxThreads () const noexcept |
| get the maximum number of threads that can be used by the refitter. More...
|
|
bool | setNamedWeights (char const *name, Weights weights, TensorLocation location) noexcept |
| Specify new weights on a specified device of given name. More...
|
|
Weights | getNamedWeights (char const *weightsName) const noexcept |
| Get weights associated with the given name. More...
|
|
TensorLocation | getWeightsLocation (char const *weightsName) const noexcept |
| Get location for the weights associated with the given name. More...
|
|
bool | unsetNamedWeights (char const *weightsName) noexcept |
| Unset weights associated with the given name. More...
|
|
void | setWeightsValidation (bool weightsValidation) noexcept |
| Set whether to validate weights during refitting. More...
|
|
bool | getWeightsValidation () const noexcept |
| Get whether to validate weights values during refitting. More...
|
|
bool | refitCudaEngineAsync (cudaStream_t stream) noexcept |
| Enqueue weights refitting of the associated engine on the given stream. More...
|
|
Weights | getWeightsPrototype (char const *weightsName) const noexcept |
| Get the Weights prototype associated with the given name. More...
|
|
Updates weights in an engine.
- Warning
- Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.
int32_t nvinfer1::IRefitter::getMissing |
( |
int32_t |
size, |
|
|
char const ** |
layerNames, |
|
|
WeightsRole * |
roles |
|
) |
| |
|
inlinenoexcept |
Get description of missing weights.
For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.
- Parameters
-
size | The number of items that can be safely written to a non-null layerNames or roles. |
layerNames | Where to write the layer names. |
roles | Where to write the weights roles. |
- Returns
- The number of missing Weights.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
int32_t nvinfer1::IRefitter::getMissingWeights |
( |
int32_t |
size, |
|
|
char const ** |
weightsNames |
|
) |
| |
|
inlinenoexcept |
Get names of missing weights.
For example, if some Weights have been set, but the engine was optimized in a way that combines weights, any unsupplied Weights in the combination are considered missing.
- Parameters
-
size | The number of weights names that can be safely written to. |
weightsNames | The names of the weights to be updated, or nullptr for unnamed weights. |
- Returns
- The number of missing Weights.
If layerNames!=nullptr, each written pointer points to a string owned by the engine being refit, and becomes invalid when the engine is destroyed.
bool nvinfer1::IRefitter::refitCudaEngineAsync |
( |
cudaStream_t |
stream | ) |
|
|
inlinenoexcept |
Enqueue weights refitting of the associated engine on the given stream.
- Parameters
-
stream | The stream to enqueue the weights updating task. |
- Returns
- True on success, or false if new weights validation fails or getMissingWeights() != 0 before the call. If false is returned, a subset of weights may have been refitted.
The behavior is undefined if the engine has pending enqueued work on a different stream from the provided one. Provided weights on CPU can be unset and released, or updated after refitCudaEngineAsync returns. Freeing or updating of the provided weights on GPU can be enqueued on the same stream after refitCudaEngineAsync returns.
IExecutionContexts associated with the engine remain valid for use afterwards. There is no need to set the same weights repeatedly for multiple refit calls as the weights memory can be updated directly instead. The weights updating task should use the same stream as the one used for the refit call.
void nvinfer1::IRefitter::setErrorRecorder |
( |
IErrorRecorder * |
recorder | ) |
|
|
inlinenoexcept |
Set the ErrorRecorder for this interface.
Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.
If an error recorder is not set, messages will be sent to the global log stream.
- Parameters
-
recorder | The error recorder to register with this interface. |
- See also
- getErrorRecorder()