Functionally safe context for executing inference using an engine.
More...
#include <NvInferSafeRuntime.h>
Functionally safe context for executing inference using an engine.
Multiple safe execution contexts may exist for one safe::ICudaEngine instance, allowing the same engine to be used for the execution of multiple inputs simultaneously.
- Warning
- Do not call the APIs of the same IExecutionContext from multiple threads at any given time. Each concurrent execution must have its own instance of an IExecutionContext.
-
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.
◆ IExecutionContext() [1/3]
| nvinfer1::safe::IExecutionContext::IExecutionContext |
( |
| ) |
|
|
default |
◆ ~IExecutionContext()
| virtual nvinfer1::safe::IExecutionContext::~IExecutionContext |
( |
| ) |
|
|
virtualdefaultnoexcept |
◆ IExecutionContext() [2/3]
◆ IExecutionContext() [3/3]
◆ enqueueV2()
| virtual bool nvinfer1::safe::IExecutionContext::enqueueV2 |
( |
void *const *const |
bindings, |
|
|
cudaStream_t const |
stream, |
|
|
cudaEvent_t *const |
inputConsumed |
|
) |
| |
|
pure virtualnoexcept |
Asynchronously execute inference on a batch.
This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using safe::ICudaEngine::getBindingIndex(). This method only works for an execution context built from a network without an implicit batch dimension.
- Parameters
-
| bindings | An array of pointers to input and output buffers for the network. |
| stream | A cuda stream on which the inference kernels will be enqueued. |
| inputConsumed | An optional event which will be signaled when the input buffers can be refilled with new data. |
- Returns
- True if the kernels were enqueued successfully.
- See also
- safe::ICudaEngine::getBindingIndex()
Usage considerations
- Allowed context for the API call
◆ getEngine()
| virtual const ICudaEngine & nvinfer1::safe::IExecutionContext::getEngine |
( |
| ) |
const |
|
pure virtualnoexcept |
Get the associated engine.
- See also
- safe::ICudaEngine
Usage considerations
- Allowed context for the API call
◆ getErrorBuffer()
Get error buffer output for floating point errors.
- Returns
- Pointer to device memory to use as floating point error buffer or nullptr if not set.
- See also
- setErrorBuffer()
Usage considerations
- Allowed context for the API call
◆ getErrorRecorder()
| virtual IErrorRecorder * nvinfer1::safe::IExecutionContext::getErrorRecorder |
( |
| ) |
const |
|
pure virtualnoexcept |
get the ErrorRecorder assigned to this interface.
Retrieves the assigned error recorder object for the given class. A default error recorder does not exist, so a nullptr will be returned if setErrorRecorder has not been called.
- Returns
- A pointer to the IErrorRecorder object that has been registered.
- See also
- setErrorRecorder()
Usage considerations
- Allowed context for the API call
◆ getName()
| virtual AsciiChar const * nvinfer1::safe::IExecutionContext::getName |
( |
| ) |
const |
|
pure virtualnoexcept |
Return the name of the execution context.
- See also
- setName()
Usage considerations
- Allowed context for the API call
◆ getStrides()
| virtual Dims nvinfer1::safe::IExecutionContext::getStrides |
( |
std::int32_t const |
bindingIndex | ) |
const |
|
pure virtualnoexcept |
Return the strides of the buffer for the given binding.
- Parameters
-
| bindingIndex | The binding index. |
Usage considerations
- Allowed context for the API call
◆ operator=() [1/2]
◆ operator=() [2/2]
◆ setDeviceMemory()
| virtual void nvinfer1::safe::IExecutionContext::setDeviceMemory |
( |
void *const |
memory | ) |
|
|
pure virtualnoexcept |
◆ setErrorBuffer()
Set error buffer output for floating point errors.
The error buffer output must be allocated in device memory and will be used for subsequent calls to enqueueV2. Checking the contents of the error buffer after inference is the responsibility of the application. The pointer passed here must have alignment adequate for the FloatingPointErrorInformation struct.
- Warning
- Do not release or use the contents of the error buffer for any other purpose before synchronizing on the CUDA stream passed to enqueueV2.
- Parameters
-
| buffer | The device memory to use as floating point error buffer |
- See also
- getErrorBuffer()
Usage considerations
- Allowed context for the API call
◆ setErrorRecorder()
| virtual void nvinfer1::safe::IExecutionContext::setErrorRecorder |
( |
IErrorRecorder *const |
recorder | ) |
|
|
pure virtualnoexcept |
Set the ErrorRecorder for this interface.
Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.
- Parameters
-
| recorder | The error recorder to register with this interface. |
- See also
- getErrorRecorder()
Usage considerations
- Allowed context for the API call
◆ setName()
| virtual void nvinfer1::safe::IExecutionContext::setName |
( |
AsciiChar const *const |
name | ) |
|
|
pure virtualnoexcept |
Set the name of the execution context.
This method copies the name string.
- Warning
- Strings passed to the runtime must be 1024 characters or less including NULL terminator and must be NULL terminated.
- See also
- getName()
Usage considerations
- Allowed context for the API call
The documentation for this class was generated from the following file:
1.9.2