nvinfer1::v_1_0::IGpuAsyncAllocator Class Reference

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::v_1_0::IGpuAsyncAllocator:

Public Member Functions
	IGpuAsyncAllocator ()=default
	~IGpuAsyncAllocator () override=default
void *	allocateAsync (uint64_t const size, uint64_t const alignment, AllocatorFlags const flags, cudaStream_t) noexcept override=0
	A thread-safe callback implemented by the application to handle stream-ordered asynchronous acquisition of GPU memory. More...
bool	deallocateAsync (void *const memory, cudaStream_t) noexcept override=0
	A thread-safe callback implemented by the application to handle stream-ordered asynchronous release of GPU memory. More...
TRT_DEPRECATED void *	allocate (uint64_t const size, uint64_t const alignment, AllocatorFlags const flags) noexcept override
	A thread-safe callback implemented by the application to handle acquisition of GPU memory. More...
TRT_DEPRECATED bool	deallocate (void *const memory) noexcept override
	A thread-safe callback implemented by the application to handle release of GPU memory. More...
InterfaceInfo	getInterfaceInfo () const noexcept override
	Return version information associated with this interface. Applications must not override this method. More...
Public Member Functions inherited from nvinfer1::v_1_0::IGpuAllocator
	~IGpuAllocator () override=default
	IGpuAllocator ()=default
virtual void *	reallocate (void *const, uint64_t, uint64_t) noexcept
	A thread-safe callback implemented by the application to resize an existing allocation. More...
InterfaceInfo	getInterfaceInfo () const noexcept override
	Return version information associated with this interface. Applications must not override this method. More...
Public Member Functions inherited from nvinfer1::IVersionedInterface
virtual APILanguage	getAPILanguage () const noexcept
	The language used to build the implementation of this Interface. More...
virtual	~IVersionedInterface () noexcept=default

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::IVersionedInterface
	IVersionedInterface ()=default
	IVersionedInterface (IVersionedInterface const &)=default
	IVersionedInterface (IVersionedInterface &&)=default
IVersionedInterface &	operator= (IVersionedInterface const &) &=default
IVersionedInterface &	operator= (IVersionedInterface &&) &=default

Constructor & Destructor Documentation

IGpuAsyncAllocator()

nvinfer1::v_1_0::IGpuAsyncAllocator::IGpuAsyncAllocator ( )

default

~IGpuAsyncAllocator()

nvinfer1::v_1_0::IGpuAsyncAllocator::~IGpuAsyncAllocator ( )

overridedefault

Member Function Documentation

allocate()

TRT_DEPRECATED void * nvinfer1::v_1_0::IGpuAsyncAllocator::allocate ( uint64_t const  size, uint64_t const  alignment, AllocatorFlags const  flags  )

inlineoverridevirtualnoexcept

A thread-safe callback implemented by the application to handle acquisition of GPU memory.

Parameters

size	The size of the memory block required (in bytes).
alignment	The required alignment of memory. Alignment will be zero or a power of 2 not exceeding the alignment guaranteed by cudaMalloc. Thus this allocator can be safely implemented with cudaMalloc/cudaFree. An alignment value of zero indicates any alignment is acceptable.
flags	Reserved for future use. In the current release, 0 will be passed.

Returns

If the allocation was successful, the start address of a device memory block of the requested size. If an allocation request of size 0 is made, nullptr must be returned. If an allocation request cannot be satisfied, nullptr must be returned. If a non-null address is returned, it is guaranteed to have the specified alignment.

Note

The implementation must guarantee thread safety for concurrent allocateAsync/deallocateAsync/reallocate requests.

Usage considerations

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

    Deprecated:

    Deprecated in TensorRT 10.0. Superseded by allocateAsync

Implements nvinfer1::v_1_0::IGpuAllocator.

allocateAsync()

void * nvinfer1::v_1_0::IGpuAsyncAllocator::allocateAsync ( uint64_t const  size, uint64_t const  alignment, AllocatorFlags const  flags, cudaStream_t    )

overridepure virtualnoexcept

A thread-safe callback implemented by the application to handle stream-ordered asynchronous acquisition of GPU memory.

Parameters

size	The size of the memory block required (in bytes).
alignment	The required alignment of memory. Alignment will be zero or a power of 2 not exceeding the alignment guaranteed by cudaMalloc. Thus this allocator can be safely implemented with cudaMalloc/cudaFree. An alignment value of zero indicates any alignment is acceptable.
flags	Reserved for future use. In the current release, 0 will be passed.
stream	Specifies the cudastream for the asynchronous allocation. If nullptr or 0 is passed, the default stream will be used.

Returns

If the allocation was successful, the start address of a device memory block of the requested size. If an allocation request of size 0 is made, nullptr must be returned. If an allocation request cannot be satisfied, nullptr must be returned. If a non-null address is returned, it is guaranteed to have the specified alignment.

Note

The implementation must guarantee thread safety for concurrent allocateAsync/deallocateAsync requests.

The implementation is not required to be asynchronous. It is permitted to synchronize, albeit doing so will lose the performance advantage of asynchronous allocation.

Usage considerations

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

Reimplemented from nvinfer1::v_1_0::IGpuAllocator.

deallocate()

TRT_DEPRECATED bool nvinfer1::v_1_0::IGpuAsyncAllocator::deallocate ( void *const  memory )

inlineoverridevirtualnoexcept

A thread-safe callback implemented by the application to handle release of GPU memory.

TensorRT may pass a nullptr to this function if it was previously returned by allocate().

Parameters

memory

A memory address that was previously returned by an allocate() or reallocate() call of the same allocator object.

Returns

True if the acquired memory is released successfully.

Note

The implementation must guarantee thread safety for concurrent allocate/reallocate/deallocate requests.

Usage considerations

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

    Deprecated:

    Deprecated in TensorRT 10.0. Superseded by deallocateAsync

Implements nvinfer1::v_1_0::IGpuAllocator.

deallocateAsync()

bool nvinfer1::v_1_0::IGpuAsyncAllocator::deallocateAsync ( void *const  memory, cudaStream_t    )

overridepure virtualnoexcept

A thread-safe callback implemented by the application to handle stream-ordered asynchronous release of GPU memory.

TensorRT may pass a nullptr to this function if it was previously returned by allocate().

Parameters

memory	A memory address that was previously returned by an allocate() or reallocate() call of the same allocator object.
stream	Specifies the cudastream for the asynchronous deallocation. If nullptr or 0 is passed, the default stream will be used.

Returns

True if the acquired memory is released successfully.

Note

The implementation must guarantee thread safety for concurrent allocateAsync/deallocateAsync requests.

The implementation is not required to be asynchronous. It is permitted to synchronize, albeit doing so will lose the performance advantage of asynchronous deallocation. Either way, it is critical that it not actually free the memory until the current stream position is reached.

Usage considerations

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

Reimplemented from nvinfer1::v_1_0::IGpuAllocator.

getInterfaceInfo()

InterfaceInfo nvinfer1::v_1_0::IGpuAsyncAllocator::getInterfaceInfo ( ) const

inlineoverridevirtualnoexcept

Return version information associated with this interface. Applications must not override this method.

Implements nvinfer1::IVersionedInterface.

---

The documentation for this class was generated from the following file:

• NvInferRuntime.h