TensorRT 8.5.3
nvinfer1::IOutputAllocator Class Referenceabstract

Callback from ExecutionContext::enqueueV3() More...

#include <NvInferRuntime.h>

Public Member Functions

virtual int32_t getInterfaceVersion () const noexcept
 Return the API version of this IOutputAllocator. More...
 
virtual void * reallocateOutput (char const *tensorName, void *currentMemory, uint64_t size, uint64_t alignment) noexcept=0
 Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. More...
 
virtual void notifyShape (char const *tensorName, Dims const &dims) noexcept=0
 Called by TensorRT when the shape of the output tensor is known. More...
 
virtual ~IOutputAllocator ()=default
 

Detailed Description

Callback from ExecutionContext::enqueueV3()

Clients should override the method reallocateOutput.

See also
IExecutionContext::enqueueV3()

Constructor & Destructor Documentation

◆ ~IOutputAllocator()

virtual nvinfer1::IOutputAllocator::~IOutputAllocator ( )
virtualdefault

Member Function Documentation

◆ getInterfaceVersion()

virtual int32_t nvinfer1::IOutputAllocator::getInterfaceVersion ( ) const
inlinevirtualnoexcept

Return the API version of this IOutputAllocator.

Do not override this method as it is used by the TensorRT library to maintain backwards-compatibility with IOutputAllocator. The value will change if Nvidia adds additional virtual methods to this class.

◆ notifyShape()

virtual void nvinfer1::IOutputAllocator::notifyShape ( char const *  tensorName,
Dims const &  dims 
)
pure virtualnoexcept

Called by TensorRT when the shape of the output tensor is known.

Called by TensorRT sometime between when it calls reallocateOutput and enqueueV3 returns.

Parameters
dimsdimensions of the output
tensorNamename of the tensor

◆ reallocateOutput()

virtual void * nvinfer1::IOutputAllocator::reallocateOutput ( char const *  tensorName,
void *  currentMemory,
uint64_t  size,
uint64_t  alignment 
)
pure virtualnoexcept

Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated.

Parameters
tensorNamename of the output tensor.
currentMemorypoints to the address set by IExectionContext::setTensorAddress.
sizenumber of bytes required. Always positive, even for an empty tensor.
alignmentrequired alignment of the allocation.
Returns
A pointer to memory to use for the output tensor or nullptr.

If currentMemory is known to be big enough, one option is to return currentMemory.

To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutput return nullptr if that memory is not big enough.


The documentation for this class was generated from the following file: