nvinfer1::v_1_0::IOutputAllocator Class Reference

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::v_1_0::IOutputAllocator:

Public Member Functions
InterfaceInfo	getInterfaceInfo () const noexcept override
	Return version information associated with this interface. Applications must not override this method. More...
virtual TRT_DEPRECATED void *	reallocateOutput (char const *tensorName, void *currentMemory, uint64_t size, uint64_t alignment) noexcept
	Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory. More...
virtual void *	reallocateOutputAsync (char const *tensorName, void *currentMemory, uint64_t size, uint64_t alignment, cudaStream_t)
	Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory. More...
virtual void	notifyShape (char const *tensorName, Dims const &dims) noexcept=0
	Called by TensorRT when the shape of the output tensor is known. More...
Public Member Functions inherited from nvinfer1::IVersionedInterface
virtual APILanguage	getAPILanguage () const noexcept
	The language used to build the implementation of this Interface. More...
virtual	~IVersionedInterface () noexcept=default

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::IVersionedInterface
	IVersionedInterface ()=default
	IVersionedInterface (IVersionedInterface const &)=default
	IVersionedInterface (IVersionedInterface &&)=default
IVersionedInterface &	operator= (IVersionedInterface const &) &=default
IVersionedInterface &	operator= (IVersionedInterface &&) &=default

Member Function Documentation

getInterfaceInfo()

InterfaceInfo nvinfer1::v_1_0::IOutputAllocator::getInterfaceInfo ( ) const

inlineoverridevirtualnoexcept

Return version information associated with this interface. Applications must not override this method.

Implements nvinfer1::IVersionedInterface.

notifyShape()

virtual void nvinfer1::v_1_0::IOutputAllocator::notifyShape ( char const *  tensorName, Dims const &  dims  )

pure virtualnoexcept

Called by TensorRT when the shape of the output tensor is known.

Called by TensorRT sometime between when it calls reallocateOutput and enqueueV3 returns.

Parameters

dims	dimensions of the output
tensorName	name of the tensor

reallocateOutput()

virtual TRT_DEPRECATED void * nvinfer1::v_1_0::IOutputAllocator::reallocateOutput ( char const *  tensorName, void *  currentMemory, uint64_t  size, uint64_t  alignment  )

inlinevirtualnoexcept

Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory.

Parameters

tensorName	name of the output tensor.
currentMemory	points to the address set by IExectionContext::setTensorAddress.
size	number of bytes required. Always positive, even for an empty tensor.
alignment	required alignment of the allocation.

Returns

A pointer to memory to use for the output tensor or nullptr.

To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutput return nullptr if that memory is not big enough.

Deprecated:

Deprecated in TensorRT 10.0. Superseded by reallocateOutputAsync with cudaStream_t argument

reallocateOutputAsync()

virtual void * nvinfer1::v_1_0::IOutputAllocator::reallocateOutputAsync ( char const *  tensorName, void *  currentMemory, uint64_t  size, uint64_t  alignment, cudaStream_t    )

inlinevirtual

Return a pointer to memory for an output tensor, or nullptr if memory cannot be allocated. If the requested memory size exceeds the currentMemory size, the currentMemory can be freed as well. If currentMemory is known to be big enough, one option is to return currentMemory.

Parameters

tensorName	name of the output tensor.
currentMemory	points to the address set by IExectionContext::setTensorAddress.
size	number of bytes required. Always positive, even for an empty tensor.
alignment	required alignment of the allocation.
stream	The stream in which to execute the kernels.

Returns

A pointer to memory to use for the output tensor or nullptr.

To preallocate memory and have the engine fail if the preallocation is not big enough, use IExecutionContext::setTensorAddress to set a pointer to the preallocated memory, and have reallocateOutputAsync return nullptr if that memory is not big enough.

The default definition exists for sake of backward compatibility with earlier versions of TensorRT. Eventually this method will become a pure virtual method that requires an override, and method reallocateOutput() will disappear. Code moving away from TensorRT 9.x should override method reallocateOutputAsync() and NOT override method reallocateOutput().

---

The documentation for this class was generated from the following file:

• NvInferRuntime.h