Allows a serialized functionally unsafe engine to be deserialized. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRuntime:

Public Member Functions
virtual	~IRuntime () noexcept=default

TRT_DEPRECATED nvinfer1::ICudaEngine *	deserializeCudaEngine (void const blob, std::size_t size, IPluginFactory pluginFactory) noexcept
	Deserialize an engine from a stream. More...

void	setDLACore (int32_t dlaCore) noexcept
	Sets the DLA core used by the network. Defaults to -1. More...

int32_t	getDLACore () const noexcept
	Get the DLA core that the engine executes on. More...

int32_t	getNbDLACores () const noexcept
	Returns number of DLA hardware cores accessible or 0 if DLA is unavailable. More...

TRT_DEPRECATED void	destroy () noexcept
	Destroy this object. More...

void	setGpuAllocator (IGpuAllocator *allocator) noexcept
	Set the GPU allocator. More...

void	setErrorRecorder (IErrorRecorder *recorder) noexcept
	Set the ErrorRecorder for this interface. More...

IErrorRecorder *	getErrorRecorder () const noexcept
	get the ErrorRecorder assigned to this interface. More...

ICudaEngine *	deserializeCudaEngine (void const *blob, std::size_t size) noexcept
	Deserialize an engine from a stream. More...

ILogger *	getLogger () const noexcept
	get the logger with which the runtime was created More...

bool	setMaxThreads (int32_t maxThreads) noexcept
	Set the maximum number of threads. More...

int32_t	getMaxThreads () const noexcept
	Get the maximum number of threads that can be used by the runtime. More...

Protected Attributes
apiv::VRuntime *	mImpl

Additional Inherited Members
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default

virtual	~INoCopy ()=default

	INoCopy (INoCopy const &other)=delete

INoCopy &	operator= (INoCopy const &other)=delete

	INoCopy (INoCopy &&other)=delete

INoCopy &	operator= (INoCopy &&other)=delete

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRuntime()

virtual nvinfer1::IRuntime::~IRuntime ( )

virtualdefaultnoexcept

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine	(	void const *	blob,
		std::size_t	size
	)

inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory.

Returns: The engine, or nullptr if it could not be deserialized.

◆ deserializeCudaEngine() [2/2]

TRT_DEPRECATED nvinfer1::ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine	(	void const *	blob,
		std::size_t	size,
		IPluginFactory *	pluginFactory
	)

inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters

blob	The memory that holds the serialized engine.
size	The size of the memory in bytes.
pluginFactory	The plugin factory, if any plugins are used by the network, otherwise nullptr.

Returns: The engine, or nullptr if it could not be deserialized.

Deprecated:: Deprecated in TensorRT 8.0.

Warning: IPluginFactory is no longer supported, therefore pluginFactory must be a nullptr.

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRuntime::destroy ( )

inlinenoexcept

Destroy this object.

Deprecated:: Deprecated in TRT 8.0. Superseded by delete.

Warning: Calling destroy on a managed pointer will result in a double-free error.

◆ getDLACore()

int32_t nvinfer1::IRuntime::getDLACore ( ) const

inlinenoexcept

Get the DLA core that the engine executes on.

Returns: assigned DLA core or -1 for DLA not present or unset.

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IRuntime::getErrorRecorder ( ) const

inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns: A pointer to the IErrorRecorder object that has been registered.

See also: setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IRuntime::getLogger ( ) const

inlinenoexcept

get the logger with which the runtime was created

Returns: the logger

◆ getMaxThreads()

int32_t nvinfer1::IRuntime::getMaxThreads ( ) const

inlinenoexcept

Get the maximum number of threads that can be used by the runtime.

Retrieves the maximum number of threads that can be used by the runtime.

Returns: The maximum number of threads that can be used by the runtime.

See also: setMaxThreads()

◆ getNbDLACores()

int32_t nvinfer1::IRuntime::getNbDLACores ( ) const

inlinenoexcept

Returns number of DLA hardware cores accessible or 0 if DLA is unavailable.

◆ setDLACore()

void nvinfer1::IRuntime::setDLACore ( int32_t dlaCore )

inlinenoexcept

Sets the DLA core used by the network. Defaults to -1.

Parameters

dlaCore The DLA core to execute the engine on, in the range [0,getNbDlaCores()).

This function is used to specify which DLA core to use via indexing, if multiple DLA cores are available.

Warning: if getNbDLACores() returns 0, then this function does nothing.

See also: getDLACore()

◆ setErrorRecorder()

void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder * recorder )

inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters

recorder The error recorder to register with this interface.

See also: getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator * allocator )

inlinenoexcept

Set the GPU allocator.

Parameters

allocator Set the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.

◆ setMaxThreads()

bool nvinfer1::IRuntime::setMaxThreads ( int32_t maxThreads )

inlinenoexcept

Set the maximum number of threads.

Parameters

maxThreads The maximum number of threads that can be used by the runtime.

Returns: True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

Member Data Documentation

◆ mImpl

apiv::VRuntime* nvinfer1::IRuntime::mImpl

protected

The documentation for this class was generated from the following file:

NvInferRuntime.h

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ ~IRuntime()

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

◆ deserializeCudaEngine() [2/2]

◆ destroy()

◆ getDLACore()

◆ getErrorRecorder()

◆ getLogger()

◆ getMaxThreads()

◆ getNbDLACores()

◆ setDLACore()

◆ setErrorRecorder()

◆ setGpuAllocator()

◆ setMaxThreads()

Member Data Documentation

◆ mImpl