Allows a serialized functionally unsafe engine to be deserialized. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRuntime:

Public Member Functions
virtual	~IRuntime () noexcept=default

TRT_DEPRECATED nvinfer1::ICudaEngine *	deserializeCudaEngine (const void blob, std::size_t size, IPluginFactory pluginFactory) noexcept
	Deserialize an engine from a stream. More...

void	setDLACore (int32_t dlaCore) noexcept
	Set the DLA core that the deserialized engine must execute on. More...

int32_t	getDLACore () const noexcept
	Get the DLA core that the engine executes on. More...

int32_t	getNbDLACores () const noexcept
	Returns number of DLA hardware cores accessible. More...

TRT_DEPRECATED void	destroy () noexcept
	Destroy this object. More...

void	setGpuAllocator (IGpuAllocator *allocator) noexcept
	Set the GPU allocator. More...

void	setErrorRecorder (IErrorRecorder *recorder) noexcept
	Set the ErrorRecorder for this interface. More...

IErrorRecorder *	getErrorRecorder () const noexcept
	get the ErrorRecorder assigned to this interface. More...

ICudaEngine *	deserializeCudaEngine (const void *blob, std::size_t size) noexcept
	Deserialize an engine from a stream. More...

ILogger *	getLogger () const noexcept
	get the logger with which the runtime was created More...

bool	setMaxThreads (int32_t maxThreads) noexcept
	Set the maximum number of threads. More...

int32_t	getMaxThreads () const noexcept
	Get the maximum number of threads that can be used by the runtime. More...

Protected Attributes
apiv::VRuntime *	mImpl

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default

virtual	~INoCopy ()=default

	INoCopy (const INoCopy &other)=delete

INoCopy &	operator= (const INoCopy &other)=delete

	INoCopy (INoCopy &&other)=delete

INoCopy &	operator= (INoCopy &&other)=delete

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRuntime()

virtual nvinfer1::IRuntime::~IRuntime ( )

virtualdefaultnoexcept

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine	(	const void *	blob,
		std::size_t	size
	)

inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory.

Returns: The engine, or nullptr if it could not be deserialized.

◆ deserializeCudaEngine() [2/2]

TRT_DEPRECATED nvinfer1::ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine	(	const void *	blob,
		std::size_t	size,
		IPluginFactory *	pluginFactory
	)

inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory in bytes.
pluginFactory	The plugin factory, if any plugins are used by the network, otherwise nullptr.

Returns: The engine, or nullptr if it could not be deserialized.

Deprecated:: Deprecated in TensorRT 8.0.

Warning: IPluginFactory is no longer supported, therefore pluginFactory must be a nullptr.

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRuntime::destroy ( )

inlinenoexcept

Destroy this object.

Deprecated:: Use delete instead. Deprecated in TRT 8.0.

Warning: Calling destroy on a managed pointer will result in a double-free error.

◆ getDLACore()

int32_t nvinfer1::IRuntime::getDLACore ( ) const

inlinenoexcept

Get the DLA core that the engine executes on.

Returns: If setDLACore is called, returns DLA core from 0 to N-1, else returns 0.

Warning: Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IRuntime::getErrorRecorder ( ) const

inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns: A pointer to the IErrorRecorder object that has been registered.

See also: setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IRuntime::getLogger ( ) const

inlinenoexcept

get the logger with which the runtime was created

Returns: the logger

◆ getMaxThreads()

int32_t nvinfer1::IRuntime::getMaxThreads ( ) const

inlinenoexcept

Get the maximum number of threads that can be used by the runtime.

Retrieves the maximum number of threads that can be used by the runtime.

Returns: The maximum number of threads that can be used by the runtime.

See also: setMaxThreads()

◆ getNbDLACores()

int32_t nvinfer1::IRuntime::getNbDLACores ( ) const

inlinenoexcept

Returns number of DLA hardware cores accessible.

◆ setDLACore()

void nvinfer1::IRuntime::setDLACore ( int32_t dlaCore )

inlinenoexcept

Set the DLA core that the deserialized engine must execute on.

Parameters

dlaCore The DLA core to execute the engine on (0 to N-1, where N is the maximum number of DLA's present on the device). Default value is 0.

See also: getDLACore()

Warning: Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ setErrorRecorder()

void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder * recorder )

inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters

recorder The error recorder to register with this interface.

See also: getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator * allocator )

inlinenoexcept

Set the GPU allocator.

Parameters

allocator Set the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.

◆ setMaxThreads()

bool nvinfer1::IRuntime::setMaxThreads ( int32_t maxThreads )

inlinenoexcept

Set the maximum number of threads.

Parameters

maxThreads The maximum number of threads that can be used by the runtime.

Returns: True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

Member Data Documentation

◆ mImpl

apiv::VRuntime* nvinfer1::IRuntime::mImpl

protected

The documentation for this class was generated from the following file:

NvInferRuntime.h

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ ~IRuntime()

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

◆ deserializeCudaEngine() [2/2]

◆ destroy()

◆ getDLACore()

◆ getErrorRecorder()

◆ getLogger()

◆ getMaxThreads()

◆ getNbDLACores()

◆ setDLACore()

◆ setErrorRecorder()

◆ setGpuAllocator()

◆ setMaxThreads()

Member Data Documentation

◆ mImpl