TensorRT 8.4.1
nvinfer1::IRuntime Class Reference

Allows a serialized functionally unsafe engine to be deserialized. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRuntime:
nvinfer1::INoCopy

Public Member Functions

virtual ~IRuntime () noexcept=default
 
TRT_DEPRECATED nvinfer1::ICudaEnginedeserializeCudaEngine (void const *blob, std::size_t size, IPluginFactory *pluginFactory) noexcept
 Deserialize an engine from a stream. More...
 
void setDLACore (int32_t dlaCore) noexcept
 Sets the DLA core used by the network. Defaults to -1. More...
 
int32_t getDLACore () const noexcept
 Get the DLA core that the engine executes on. More...
 
int32_t getNbDLACores () const noexcept
 Returns number of DLA hardware cores accessible or 0 if DLA is unavailable. More...
 
TRT_DEPRECATED void destroy () noexcept
 Destroy this object. More...
 
void setGpuAllocator (IGpuAllocator *allocator) noexcept
 Set the GPU allocator. More...
 
void setErrorRecorder (IErrorRecorder *recorder) noexcept
 Set the ErrorRecorder for this interface. More...
 
IErrorRecordergetErrorRecorder () const noexcept
 get the ErrorRecorder assigned to this interface. More...
 
ICudaEnginedeserializeCudaEngine (void const *blob, std::size_t size) noexcept
 Deserialize an engine from a stream. More...
 
ILoggergetLogger () const noexcept
 get the logger with which the runtime was created More...
 
bool setMaxThreads (int32_t maxThreads) noexcept
 Set the maximum number of threads. More...
 
int32_t getMaxThreads () const noexcept
 Get the maximum number of threads that can be used by the runtime. More...
 

Protected Attributes

apiv::VRuntime * mImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy ()=default
 
virtual ~INoCopy ()=default
 
 INoCopy (INoCopy const &other)=delete
 
INoCopyoperator= (INoCopy const &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRuntime()

virtual nvinfer1::IRuntime::~IRuntime ( )
virtualdefaultnoexcept

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine ( void const *  blob,
std::size_t  size 
)
inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory.
Returns
The engine, or nullptr if it could not be deserialized.

◆ deserializeCudaEngine() [2/2]

TRT_DEPRECATED nvinfer1::ICudaEngine * nvinfer1::IRuntime::deserializeCudaEngine ( void const *  blob,
std::size_t  size,
IPluginFactory *  pluginFactory 
)
inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory in bytes.
pluginFactoryThe plugin factory, if any plugins are used by the network, otherwise nullptr.
Returns
The engine, or nullptr if it could not be deserialized.
Deprecated:
Deprecated in TensorRT 8.0.
Warning
IPluginFactory is no longer supported, therefore pluginFactory must be a nullptr.

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRuntime::destroy ( )
inlinenoexcept

Destroy this object.

Deprecated:
Use delete instead. Deprecated in TRT 8.0.
Warning
Calling destroy on a managed pointer will result in a double-free error.

◆ getDLACore()

int32_t nvinfer1::IRuntime::getDLACore ( ) const
inlinenoexcept

Get the DLA core that the engine executes on.

Returns
assigned DLA core or -1 for DLA not present or unset.

◆ getErrorRecorder()

IErrorRecorder * nvinfer1::IRuntime::getErrorRecorder ( ) const
inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()

◆ getLogger()

ILogger * nvinfer1::IRuntime::getLogger ( ) const
inlinenoexcept

get the logger with which the runtime was created

Returns
the logger

◆ getMaxThreads()

int32_t nvinfer1::IRuntime::getMaxThreads ( ) const
inlinenoexcept

Get the maximum number of threads that can be used by the runtime.

Retrieves the maximum number of threads that can be used by the runtime.

Returns
The maximum number of threads that can be used by the runtime.
See also
setMaxThreads()

◆ getNbDLACores()

int32_t nvinfer1::IRuntime::getNbDLACores ( ) const
inlinenoexcept

Returns number of DLA hardware cores accessible or 0 if DLA is unavailable.

◆ setDLACore()

void nvinfer1::IRuntime::setDLACore ( int32_t  dlaCore)
inlinenoexcept

Sets the DLA core used by the network. Defaults to -1.

Parameters
dlaCoreThe DLA core to execute the engine on, in the range [0,getNbDlaCores()).

This function is used to specify which DLA core to use via indexing, if multiple DLA cores are available.

Warning
if getNbDLACores() returns 0, then this function does nothing.
See also
getDLACore()

◆ setErrorRecorder()

void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder recorder)
inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator allocator)
inlinenoexcept

Set the GPU allocator.

Parameters
allocatorSet the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.

◆ setMaxThreads()

bool nvinfer1::IRuntime::setMaxThreads ( int32_t  maxThreads)
inlinenoexcept

Set the maximum number of threads.

Parameters
maxThreadsThe maximum number of threads that can be used by the runtime.
Returns
True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

Member Data Documentation

◆ mImpl

apiv::VRuntime* nvinfer1::IRuntime::mImpl
protected

The documentation for this class was generated from the following file: