TensorRT  8.0.2
nvinfer1::IRuntime Class Reference

Allows a serialized functionally unsafe engine to be deserialized. More...

#include <NvInferRuntime.h>

Inheritance diagram for nvinfer1::IRuntime:
nvinfer1::INoCopy

Public Member Functions

TRT_DEPRECATED nvinfer1::ICudaEnginedeserializeCudaEngine (const void *blob, std::size_t size, IPluginFactory *pluginFactory) noexcept
 Deserialize an engine from a stream. More...
 
void setDLACore (int32_t dlaCore) noexcept
 Set the DLA core that the deserialized engine must execute on. More...
 
int32_t getDLACore () const noexcept
 Get the DLA core that the engine executes on. More...
 
int32_t getNbDLACores () const noexcept
 Returns number of DLA hardware cores accessible.
 
TRT_DEPRECATED void destroy () noexcept
 Destroy this object. More...
 
void setGpuAllocator (IGpuAllocator *allocator) noexcept
 Set the GPU allocator. More...
 
void setErrorRecorder (IErrorRecorder *recorder) noexcept
 Set the ErrorRecorder for this interface. More...
 
IErrorRecordergetErrorRecorder () const noexcept
 get the ErrorRecorder assigned to this interface. More...
 
nvinfer1::ICudaEnginedeserializeCudaEngine (const void *blob, std::size_t size) noexcept
 Deserialize an engine from a stream. More...
 

Protected Attributes

apiv::VRuntimemImpl
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy (const INoCopy &other)=delete
 
INoCopyoperator= (const INoCopy &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine ( const void *  blob,
std::size_t  size 
)
inlinenoexcept

Deserialize an engine from a stream.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory.
Returns
The engine, or nullptr if it could not be deserialized.

◆ deserializeCudaEngine() [2/2]

TRT_DEPRECATED nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine ( const void *  blob,
std::size_t  size,
IPluginFactory *  pluginFactory 
)
inlinenoexcept

Deserialize an engine from a stream.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory in bytes.
pluginFactoryThe plugin factory, if any plugins are used by the network, otherwise nullptr.
Returns
The engine, or nullptr if it could not be deserialized.
Deprecated:
Deprecated interface will be removed in TensorRT 10.0.
Warning
IPluginFactory is no longer supported, therefore pluginFactory must be a nullptr.

◆ destroy()

TRT_DEPRECATED void nvinfer1::IRuntime::destroy ( )
inlinenoexcept

Destroy this object.

Deprecated:
Deprecated interface will be removed in TensorRT 10.0.
Warning
Calling destroy on a managed pointer will result in a double-free error.

◆ getDLACore()

int32_t nvinfer1::IRuntime::getDLACore ( ) const
inlinenoexcept

Get the DLA core that the engine executes on.

Returns
If setDLACore is called, returns DLA core from 0 to N-1, else returns 0.
Warning
Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ getErrorRecorder()

IErrorRecorder* nvinfer1::IRuntime::getErrorRecorder ( ) const
inlinenoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()

◆ setDLACore()

void nvinfer1::IRuntime::setDLACore ( int32_t  dlaCore)
inlinenoexcept

Set the DLA core that the deserialized engine must execute on.

Parameters
dlaCoreThe DLA core to execute the engine on (0 to N-1, where N is the maximum number of DLA's present on the device). Default value is 0.
See also
getDLACore()
Warning
Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ setErrorRecorder()

void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder recorder)
inlinenoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()

◆ setGpuAllocator()

void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator allocator)
inlinenoexcept

Set the GPU allocator.

Parameters
allocatorSet the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.


The documentation for this class was generated from the following file: