Callback from ExecutionContext::enqueueV3() More...

#include <NvInferRuntime.h>

Public Member Functions
virtual int32_t	getInterfaceVersion () const noexcept
	Return the API version of this IOutputAllocator. More...

virtual void *	reallocateOutput (char const tensorName, void currentMemory, uint64_t size, uint64_t alignment) noexcept=0
	Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. More...

virtual void	notifyShape (char const *tensorName, Dims const &dims) noexcept=0
	Called by TensorRT when the shape of the output tensor is known. More...

virtual	~IOutputAllocator ()=default

Detailed Description

Callback from ExecutionContext::enqueueV3()

Clients should override the method reallocateOutput.

See also: IExecutionContext::enqueueV3()

Constructor & Destructor Documentation

◆ ~IOutputAllocator()

virtual nvinfer1::IOutputAllocator::~IOutputAllocator ( )

virtualdefault

Member Function Documentation

◆ getInterfaceVersion()

virtual int32_t nvinfer1::IOutputAllocator::getInterfaceVersion ( ) const

inlinevirtualnoexcept

Return the API version of this IOutputAllocator.

Do not override this method as it is used by the TensorRT library to maintain backwards-compatibility with IOutputAllocator. The value will change if Nvidia adds additional virtual methods to this class.

◆ notifyShape()

virtual void nvinfer1::IOutputAllocator::notifyShape	(	char const *	tensorName,
		Dims const &	dims
	)

pure virtualnoexcept

Called by TensorRT when the shape of the output tensor is known.

Called by TensorRT sometime between when it calls reallocateOutput and enqueueV3 returns.

Parameters

dims	dimensions of the output
tensorName	name of the tensor

◆ reallocateOutput()

virtual void * nvinfer1::IOutputAllocator::reallocateOutput	(	char const *	tensorName,
		void *	currentMemory,
		uint64_t	size,
		uint64_t	alignment
	)

pure virtualnoexcept

Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated.

Parameters

tensorName	name of the output tensor.
currentMemory	points to the address set by IExectionContext::setTensorAddress.
size	number of bytes required. Always positive, even for an empty tensor.
alignment	required alignment of the allocation.

Returns: A pointer to memory to use for the output tensor or nullptr.

If currentMemory is known to be big enough, one option is to return currentMemory.

To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutput return nullptr if that memory is not big enough.

The documentation for this class was generated from the following file:

NvInferRuntime.h