nvinfer2::safe::ISafeMemAllocator Class Reference

Application-implemented class for controlling memory allocation on the GPU/CPU. More...

#include <NvInferSafeMemAllocator.h>

Public Member Functions
virtual void *	allocate (uint64_t const size, uint64_t const alignment, MemoryPlacement const flags, MemoryUsage const usage, ISafeRecorder &recorder) noexcept=0
	A thread-safe callback implemented by the application to handle acquisition of GPU/CPU memory. More...
virtual	~ISafeMemAllocator ()=default
	ISafeMemAllocator ()=default
virtual bool	deallocate (void *const memory, MemoryPlacement const flags, ISafeRecorder &recorder) noexcept=0
	A thread-safe callback implemented by the application to handle release of GPU/CPU memory. More...

Protected Member Functions
	ISafeMemAllocator (ISafeMemAllocator const &)=default
	ISafeMemAllocator (ISafeMemAllocator &&)=default
ISafeMemAllocator &	operator= (ISafeMemAllocator const &) &=default
ISafeMemAllocator &	operator= (ISafeMemAllocator &&) &=default

Detailed Description

Application-implemented class for controlling memory allocation on the GPU/CPU.

Warning

The lifetime of an ISafeMemAllocator object must exceed that of all objects that use it.

Constructor & Destructor Documentation

~ISafeMemAllocator()

virtual nvinfer2::safe::ISafeMemAllocator::~ISafeMemAllocator ( )

virtualdefault

ISafeMemAllocator() [1/3]

nvinfer2::safe::ISafeMemAllocator::ISafeMemAllocator ( )

default

ISafeMemAllocator() [2/3]

nvinfer2::safe::ISafeMemAllocator::ISafeMemAllocator ( ISafeMemAllocator const &  )

protecteddefault

ISafeMemAllocator() [3/3]

nvinfer2::safe::ISafeMemAllocator::ISafeMemAllocator ( ISafeMemAllocator &&  )

protecteddefault

Member Function Documentation

allocate()

virtual void * nvinfer2::safe::ISafeMemAllocator::allocate ( uint64_t const  size, uint64_t const  alignment, MemoryPlacement const  flags, MemoryUsage const  usage, ISafeRecorder &  recorder  )

pure virtualnoexcept

A thread-safe callback implemented by the application to handle acquisition of GPU/CPU memory.

Precondition

INIT API

Parameters

size	The size of the memory block required (in bytes).
alignment	The required alignment of memory. Alignment will be zero or a power of 2 not exceeding the alignment guaranteed by cudaMalloc. Thus this allocator can be safely implemented with cudaMalloc/cudaFree. An alignment value of zero indicates any alignment is acceptable.
flags	The type of memory to be allocated: one of MemoryPlacement::kGPU, MemoryPlacement::kCPU, MemoryPlacement::kCPU_PINNED.
usage	The intended usage of memory to be allocated: one of MemoryUsage::kIMMUTABLE, MemoryUsage::kSCRATCH, MemoryUsage::kIOTENSOR.
recorder	A safe message recorder

Returns

If the allocation was successful, the start address of a device memory block of the requested size, nullptr otherwise.

• If an allocation request of size 0 is made, nullptr must be returned.
• If an allocation request cannot be satisfied for any reason, nullptr must be returned.
• If a non-null address is returned, it is guaranteed to have the specified alignment.

Warning

The implementation must guarantee thread safety for concurrent allocate/deallocate requests.

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

deallocate()

virtual bool nvinfer2::safe::ISafeMemAllocator::deallocate ( void *const  memory, MemoryPlacement const  flags, ISafeRecorder &  recorder  )

pure virtualnoexcept

A thread-safe callback implemented by the application to handle release of GPU/CPU memory.

Precondition

INIT API

TensorRT may pass a nullptr to this function if it was previously returned by allocate().

Parameters

memory	A memory address that was previously returned by an allocate() call of the same allocator object.
flags	The type of memory to be deallocated: one of MemoryPlacement::kGPU, MemoryPlacement::kCPU, MemoryPlacement::kCPU_PINNED.
recorder	A safe message recorder

Returns

True if the acquired memory is released successfully, false otherwise.

Warning

The implementation must guarantee thread safety for concurrent allocate/deallocate requests.

Note

Calling deallocate() multiple times with the same pointer results in undefined behavior. Implementations MUST detect and reject double-free attempts.

• Allowed context for the API call
  • Thread-safe: Yes, this method is required to be thread-safe and may be called from multiple threads.

operator=() [1/2]

ISafeMemAllocator & nvinfer2::safe::ISafeMemAllocator::operator= ( ISafeMemAllocator &&  ) &

protecteddefault

operator=() [2/2]

ISafeMemAllocator & nvinfer2::safe::ISafeMemAllocator::operator= ( ISafeMemAllocator const &  ) &

protecteddefault

---

The documentation for this class was generated from the following file:

• NvInferSafeMemAllocator.h