Allows a serialized functionally unsafe engine to be deserialized. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRuntime:

Public Member Functions
TRT_DEPRECATED nvinfer1::ICudaEngine *	deserializeCudaEngine (const void blob, std::size_t size, IPluginFactory pluginFactory) noexcept
	Deserialize an engine from a stream. More...

void	setDLACore (int32_t dlaCore) noexcept
	Set the DLA core that the deserialized engine must execute on. More...

int32_t	getDLACore () const noexcept
	Get the DLA core that the engine executes on. More...

int32_t	getNbDLACores () const noexcept
	Returns number of DLA hardware cores accessible.

TRT_DEPRECATED void	destroy () noexcept
	Destroy this object. More...

void	setGpuAllocator (IGpuAllocator *allocator) noexcept
	Set the GPU allocator. More...

void	setErrorRecorder (IErrorRecorder *recorder) noexcept
	Set the ErrorRecorder for this interface. More...

IErrorRecorder *	getErrorRecorder () const noexcept
	get the ErrorRecorder assigned to this interface. More...

nvinfer1::ICudaEngine *	deserializeCudaEngine (const void *blob, std::size_t size) noexcept
	Deserialize an engine from a stream. More...

Protected Attributes
apiv::VRuntime *	mImpl

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy (const INoCopy &other)=delete

INoCopy &	operator= (const INoCopy &other)=delete

	INoCopy (INoCopy &&other)=delete

INoCopy &	operator= (INoCopy &&other)=delete

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine	(	const void *	blob,
		std::size_t	size
	)

inlinenoexcept

Deserialize an engine from a stream.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory.

Returns: The engine, or nullptr if it could not be deserialized.

◆ deserializeCudaEngine() [2/2]

TRT_DEPRECATED nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine	(	const void *	blob,
		std::size_t	size,
		IPluginFactory *	pluginFactory
	)

inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory in bytes.
pluginFactory	The plugin factory, if any plugins are used by the network, otherwise nullptr.

Returns: The engine, or nullptr if it could not be deserialized.

Deprecated:: Deprecated interface will be removed in TensorRT 10.0.

Warning: IPluginFactory is no longer supported, therefore pluginFactory must be a nullptr.

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRuntime::destroy ( )

inlinenoexcept

Destroy this object.

Deprecated:: Deprecated interface will be removed in TensorRT 10.0.

Warning: Calling destroy on a managed pointer will result in a double-free error.

◆ getDLACore()

int32_t nvinfer1::IRuntime::getDLACore ( ) const

inlinenoexcept

Get the DLA core that the engine executes on.

Returns: If setDLACore is called, returns DLA core from 0 to N-1, else returns 0.

Warning: Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ getErrorRecorder()

IErrorRecorder* nvinfer1::IRuntime::getErrorRecorder ( ) const

inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns: A pointer to the IErrorRecorder object that has been registered.

See also: setErrorRecorder()

◆ setDLACore()

void nvinfer1::IRuntime::setDLACore ( int32_t dlaCore )

inlinenoexcept

Set the DLA core that the deserialized engine must execute on.

Parameters

dlaCore The DLA core to execute the engine on (0 to N-1, where N is the maximum number of DLA's present on the device). Default value is 0.

See also: getDLACore()

Warning: Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ setErrorRecorder()

void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder * recorder )

inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters

recorder The error recorder to register with this interface.

See also: getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator * allocator )

inlinenoexcept

Set the GPU allocator.

Parameters

allocator Set the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.

The documentation for this class was generated from the following file:

NvInferRuntime.h

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

◆ deserializeCudaEngine() [2/2]

◆ destroy()

◆ getDLACore()

◆ getErrorRecorder()

◆ setDLACore()

◆ setErrorRecorder()

◆ setGpuAllocator()