TensorRT 8.5.3
nvinfer1::safe::ICudaEngine Class Referenceabstract

A functionally safe engine for executing inference on a built network. More...

#include <NvInferSafeRuntime.h>

Public Member Functions

virtual TRT_DEPRECATED std::int32_t getNbBindings () const noexcept=0
 Get the number of binding indices. More...
 
virtual TRT_DEPRECATED std::int32_t getBindingIndex (AsciiChar const *const name) const noexcept=0
 Retrieve the binding index for a named tensor. More...
 
virtual TRT_DEPRECATED AsciiChar const * getBindingName (std::int32_t const bindingIndex) const noexcept=0
 Retrieve the name corresponding to a binding index. More...
 
virtual TRT_DEPRECATED bool bindingIsInput (std::int32_t const bindingIndex) const noexcept=0
 Determine whether a binding is an input binding. More...
 
virtual TRT_DEPRECATED Dims getBindingDimensions (std::int32_t const bindingIndex) const noexcept=0
 Get the dimensions of a binding. More...
 
virtual TRT_DEPRECATED DataType getBindingDataType (std::int32_t const bindingIndex) const noexcept=0
 Determine the required data type for a buffer from its binding index. More...
 
virtual IExecutionContextcreateExecutionContext () noexcept=0
 Create an execution context. More...
 
virtual IExecutionContextcreateExecutionContextWithoutDeviceMemory () noexcept=0
 Create an execution context without any device memory allocated. More...
 
virtual size_t getDeviceMemorySize () const noexcept=0
 Return the amount of device memory required by an execution context. More...
 
virtual TRT_DEPRECATED std::int32_t getBindingBytesPerComponent (std::int32_t const bindingIndex) const noexcept=0
 Return the number of bytes per component of an element. More...
 
virtual TRT_DEPRECATED std::int32_t getBindingComponentsPerElement (std::int32_t const bindingIndex) const noexcept=0
 Return the number of components included in one element. More...
 
virtual TRT_DEPRECATED TensorFormat getBindingFormat (std::int32_t const bindingIndex) const noexcept=0
 Return the binding format. More...
 
virtual TRT_DEPRECATED std::int32_t getBindingVectorizedDim (std::int32_t const bindingIndex) const noexcept=0
 Return the dimension index that the buffer is vectorized. More...
 
virtual AsciiChar const * getName () const noexcept=0
 Returns the name of the network associated with the engine. More...
 
virtual void setErrorRecorder (IErrorRecorder *const recorder) noexcept=0
 Set the ErrorRecorder for this interface. More...
 
virtual IErrorRecordergetErrorRecorder () const noexcept=0
 Get the ErrorRecorder assigned to this interface. More...
 
 ICudaEngine ()=default
 
virtual ~ICudaEngine () noexcept=default
 
 ICudaEngine (ICudaEngine const &)=delete
 
 ICudaEngine (ICudaEngine &&)=delete
 
ICudaEngineoperator= (ICudaEngine const &) &=delete
 
ICudaEngineoperator= (ICudaEngine &&) &=delete
 
virtual Dims getTensorShape (AsciiChar const *tensorName) const noexcept=0
 Get extent of an input or output tensor. More...
 
virtual DataType getTensorDataType (AsciiChar const *tensorName) const noexcept=0
 Determine the required data type for a buffer from its tensor name. More...
 
virtual TensorIOMode getTensorIOMode (AsciiChar const *tensorName) const noexcept=0
 Determine whether a tensor is an input or output tensor. More...
 
virtual std::int32_t getTensorBytesPerComponent (AsciiChar const *tensorName) const noexcept=0
 Return the number of bytes per component of an element. More...
 
virtual std::int32_t getTensorComponentsPerElement (AsciiChar const *tensorName) const noexcept=0
 Return the number of components included in one element. More...
 
virtual TensorFormat getTensorFormat (AsciiChar const *tensorName) const noexcept=0
 Return the tensor format. More...
 
virtual std::int32_t getTensorVectorizedDim (AsciiChar const *tensorName) const noexcept=0
 Return the dimension index along which buffer is vectorized. More...
 
virtual std::int32_t getNbIOTensors () const noexcept=0
 Return the number of input and output tensors for the network from which the engine was built. More...
 
virtual AsciiChar const * getIOTensorName (std::int32_t const index) const noexcept=0
 Return the name of an IO tensor. More...
 

Detailed Description

A functionally safe engine for executing inference on a built network.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ICudaEngine() [1/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( )
default

◆ ~ICudaEngine()

virtual nvinfer1::safe::ICudaEngine::~ICudaEngine ( )
virtualdefaultnoexcept

◆ ICudaEngine() [2/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( ICudaEngine const &  )
delete

◆ ICudaEngine() [3/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( ICudaEngine &&  )
delete

Member Function Documentation

◆ bindingIsInput()

virtual TRT_DEPRECATED bool nvinfer1::safe::ICudaEngine::bindingIsInput ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Determine whether a binding is an input binding.

Parameters
bindingIndexThe binding index.
Returns
True if the index corresponds to an input binding and the index is in range.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by tensorIOMode().
See also
safe::ICudaEngine::tensorIOMode()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ createExecutionContext()

virtual IExecutionContext * nvinfer1::safe::ICudaEngine::createExecutionContext ( )
pure virtualnoexcept

Create an execution context.

See also
safe::IExecutionContext.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes; if createExecutionContext fails, users should treat this as a critical error and not perform any subsequent TensorRT operations apart from outputting the error logs.

◆ createExecutionContextWithoutDeviceMemory()

virtual IExecutionContext * nvinfer1::safe::ICudaEngine::createExecutionContextWithoutDeviceMemory ( )
pure virtualnoexcept

Create an execution context without any device memory allocated.

The memory for execution of this device context must be supplied by the application.

See also
getDeviceMemorySize() safe::IExecutionContext::setDeviceMemory()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes; if createExecutionContext fails, users should treat this as a critical error and not perform any subsequent TensorRT operations apart from outputting the error logs.

◆ getBindingBytesPerComponent()

virtual TRT_DEPRECATED std::int32_t nvinfer1::safe::ICudaEngine::getBindingBytesPerComponent ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Return the number of bytes per component of an element.

The vector component size is returned if getBindingVectorizedDim() != -1.

Parameters
bindingIndexThe binding Index.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorBytesPerComponent().
See also
safe::ICudaEngine::getTensorBytesPerComponent()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingComponentsPerElement()

virtual TRT_DEPRECATED std::int32_t nvinfer1::safe::ICudaEngine::getBindingComponentsPerElement ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Return the number of components included in one element.

The number of elements in the vectors is returned if getBindingVectorizedDim() != -1.

Parameters
bindingIndexThe binding Index.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorComponentsPerElement().
See also
safe::ICudaEngine::getTensorComponentsPerElement()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingDataType()

virtual TRT_DEPRECATED DataType nvinfer1::safe::ICudaEngine::getBindingDataType ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Determine the required data type for a buffer from its binding index.

Parameters
bindingIndexThe binding index.
Returns
The type of the data in the buffer.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorDataType().
See also
safe::ICudaEngine::getTensorDataType()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingDimensions()

virtual TRT_DEPRECATED Dims nvinfer1::safe::ICudaEngine::getBindingDimensions ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Get the dimensions of a binding.

Parameters
bindingIndexThe binding index.
Returns
The dimensions of the binding if the index is in range, otherwise Dims()
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorShape().
See also
safe::ICudaEngine::getTensorShape()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingFormat()

virtual TRT_DEPRECATED TensorFormat nvinfer1::safe::ICudaEngine::getBindingFormat ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Return the binding format.

Parameters
bindingIndexThe binding Index.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorFormat().
See also
safe::ICudaEngine::getTensorFormat()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingIndex()

virtual TRT_DEPRECATED std::int32_t nvinfer1::safe::ICudaEngine::getBindingIndex ( AsciiChar const *const  name) const
pure virtualnoexcept

Retrieve the binding index for a named tensor.

safe::IExecutionContext::enqueueV2() requires an array of buffers. Engine bindings map from tensor names to indices in this array. Binding indices are assigned at engine build time, and take values in the range [0 ... n-1] where n is the total number of inputs and outputs.

Warning
Strings passed to the runtime must be 1024 characters or less including NULL terminator and must be NULL terminated.
Parameters
nameThe tensor name.
Returns
The binding index for the named tensor, or -1 if the name is not found.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by name-based methods. Use them instead of binding-index based methods.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingName()

virtual TRT_DEPRECATED AsciiChar const * nvinfer1::safe::ICudaEngine::getBindingName ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Retrieve the name corresponding to a binding index.

This is the reverse mapping to that provided by getBindingIndex().

Parameters
bindingIndexThe binding index.
Returns
The name corresponding to the index, or nullptr if the index is out of range.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by name-based methods. Use them instead of binding-index based methods.
See also
getBindingIndex()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getBindingVectorizedDim()

virtual TRT_DEPRECATED std::int32_t nvinfer1::safe::ICudaEngine::getBindingVectorizedDim ( std::int32_t const  bindingIndex) const
pure virtualnoexcept

Return the dimension index that the buffer is vectorized.

Specifically -1 is returned if scalars per vector is 1.

Parameters
bindingIndexThe binding Index.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getTensorVectorizedDim().
See also
safe::ICudaEngine::getTensorVectorizedDim()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getDeviceMemorySize()

virtual size_t nvinfer1::safe::ICudaEngine::getDeviceMemorySize ( ) const
pure virtualnoexcept

Return the amount of device memory required by an execution context.

See also
safe::IExecutionContext::setDeviceMemory()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getErrorRecorder()

virtual IErrorRecorder * nvinfer1::safe::ICudaEngine::getErrorRecorder ( ) const
pure virtualnoexcept

Get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error reporter has not been inherited from the IRuntime, and setErrorReporter() has not been called.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getIOTensorName()

virtual AsciiChar const * nvinfer1::safe::ICudaEngine::getIOTensorName ( std::int32_t const  index) const
pure virtualnoexcept

Return the name of an IO tensor.

If the index does not fall between 0 and getNbIOTensors()-1, the function will fail with an error code of ErrorCode::kINVALID_ARGUMENT(3) that is emitted to the registered IErrorRecorder.

Parameters
indexThe value that falls between 0 and getNbIOTensors()-1.
Returns
The name of an IO tensor. nullptr will be returned if the index does not fall between 0 and getNbIOTensors()-1.
See also
getNbIOTensors()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getName()

virtual AsciiChar const * nvinfer1::safe::ICudaEngine::getName ( ) const
pure virtualnoexcept

Returns the name of the network associated with the engine.

The name is set during network creation and is retrieved after building or deserialization.

See also
INetworkDefinition::setName(), INetworkDefinition::getName()
Returns
A null-terminated C-style string representing the name of the network.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getNbBindings()

virtual TRT_DEPRECATED std::int32_t nvinfer1::safe::ICudaEngine::getNbBindings ( ) const
pure virtualnoexcept

Get the number of binding indices.

Returns
The number of binding indices.
Deprecated:
Deprecated in TensorRT 8.5. Superseded by getNbIOTensors.
See also
getBindingIndex()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getNbIOTensors()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getNbIOTensors ( ) const
pure virtualnoexcept

Return the number of input and output tensors for the network from which the engine was built.

Returns
The number of IO tensors.
See also
getIOTensorName()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorBytesPerComponent()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getTensorBytesPerComponent ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Return the number of bytes per component of an element.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
The vector component size. 0 will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit, or (4) the tensor of given name is not vectorized.
See also
safe::ICudaEngine::getTensorVectorizedDim()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorComponentsPerElement()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getTensorComponentsPerElement ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Return the number of components included in one element.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
The vector component size. -1 will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit, or (4) the tensor of given name is not vectorized.
See also
safe::ICudaEngine::getTensorVectorizedDim()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorDataType()

virtual DataType nvinfer1::safe::ICudaEngine::getTensorDataType ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Determine the required data type for a buffer from its tensor name.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
The type of the data in the buffer. DataType::kFLOAT will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorFormat()

virtual TensorFormat nvinfer1::safe::ICudaEngine::getTensorFormat ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Return the tensor format.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
The tensor format. TensorFormat::kLINEAR will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorIOMode()

virtual TensorIOMode nvinfer1::safe::ICudaEngine::getTensorIOMode ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Determine whether a tensor is an input or output tensor.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
kINPUT if tensorName is an input, kOUTPUT if tensorName is an output, or kNONE if neither.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorShape()

virtual Dims nvinfer1::safe::ICudaEngine::getTensorShape ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Get extent of an input or output tensor.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
Extent of the tensor. Dims{-1, {}} will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ getTensorVectorizedDim()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getTensorVectorizedDim ( AsciiChar const *  tensorName) const
pure virtualnoexcept

Return the dimension index along which buffer is vectorized.

Specifically -1 is returned if scalars per vector is 1.

Parameters
tensorNameThe name of an input or output tensor.
Warning
The string tensorName must be 1024 characters or less including NULL terminator and must be NULL terminated.
Returns
The dimension index along which the buffer is vectorized. -1 will be returned if (1) name is not the name of an input or output tensor, or (2) name is nullptr, or (3) name exceeds the string length limit, or (4) the tensor of given name is not vectorized.


Usage considerations

  • Allowed context for the API call
    • Thread-safe: Yes

◆ operator=() [1/2]

ICudaEngine & nvinfer1::safe::ICudaEngine::operator= ( ICudaEngine &&  ) &
delete

◆ operator=() [2/2]

ICudaEngine & nvinfer1::safe::ICudaEngine::operator= ( ICudaEngine const &  ) &
delete

◆ setErrorRecorder()

virtual void nvinfer1::safe::ICudaEngine::setErrorRecorder ( IErrorRecorder *const  recorder)
pure virtualnoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder()


Usage considerations

  • Allowed context for the API call
    • Thread-safe: No

The documentation for this class was generated from the following file: