A functionally safe engine for executing inference on a built network. More...

#include <NvInferSafeRuntime.h>

Public Member Functions
virtual std::int32_t	getNbBindings () const noexcept=0
	Get the number of binding indices. More...

virtual std::int32_t	getBindingIndex (AsciiChar const *const name) const noexcept=0
	Retrieve the binding index for a named tensor. More...

virtual AsciiChar const *	getBindingName (std::int32_t const bindingIndex) const noexcept=0
	Retrieve the name corresponding to a binding index. More...

virtual bool	bindingIsInput (std::int32_t const bindingIndex) const noexcept=0
	Determine whether a binding is an input binding. More...

virtual Dims	getBindingDimensions (std::int32_t const bindingIndex) const noexcept=0
	Get the dimensions of a binding. More...

virtual DataType	getBindingDataType (std::int32_t const bindingIndex) const noexcept=0
	Determine the required data type for a buffer from its binding index. More...

virtual IExecutionContext *	createExecutionContext () noexcept=0
	Create an execution context. More...

virtual IExecutionContext *	createExecutionContextWithoutDeviceMemory () noexcept=0
	Create an execution context without any device memory allocated. More...

virtual size_t	getDeviceMemorySize () const noexcept=0
	Return the amount of device memory required by an execution context. More...

virtual std::int32_t	getBindingBytesPerComponent (std::int32_t const bindingIndex) const noexcept=0
	Return the number of bytes per component of an element. More...

virtual std::int32_t	getBindingComponentsPerElement (std::int32_t const bindingIndex) const noexcept=0
	Return the number of components included in one element. More...

virtual TensorFormat	getBindingFormat (std::int32_t const bindingIndex) const noexcept=0
	Return the binding format. More...

virtual std::int32_t	getBindingVectorizedDim (std::int32_t const bindingIndex) const noexcept=0
	Return the dimension index that the buffer is vectorized. More...

virtual AsciiChar const *	getName () const noexcept=0
	Returns the name of the network associated with the engine. More...

virtual void	setErrorRecorder (IErrorRecorder *const recorder) noexcept=0
	Set the ErrorRecorder for this interface. More...

virtual IErrorRecorder *	getErrorRecorder () const noexcept=0
	Get the ErrorRecorder assigned to this interface. More...

	ICudaEngine ()=default

virtual	~ICudaEngine () noexcept=default

	ICudaEngine (ICudaEngine const &)=delete

	ICudaEngine (ICudaEngine &&)=delete

ICudaEngine &	operator= (ICudaEngine const &) &=delete

ICudaEngine &	operator= (ICudaEngine &&) &=delete

Detailed Description

A functionally safe engine for executing inference on a built network.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ICudaEngine() [1/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( )

default

◆ ~ICudaEngine()

virtual nvinfer1::safe::ICudaEngine::~ICudaEngine ( )

virtualdefaultnoexcept

◆ ICudaEngine() [2/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( ICudaEngine const & )

delete

◆ ICudaEngine() [3/3]

nvinfer1::safe::ICudaEngine::ICudaEngine ( ICudaEngine && )

delete

Member Function Documentation

◆ bindingIsInput()

virtual bool nvinfer1::safe::ICudaEngine::bindingIsInput ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Determine whether a binding is an input binding.

Parameters

bindingIndex The binding index.

Returns: True if the index corresponds to an input binding and the index is in range.

See also: getBindingIndex()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ createExecutionContext()

virtual IExecutionContext * nvinfer1::safe::ICudaEngine::createExecutionContext ( )

pure virtualnoexcept

Create an execution context.

See also: safe::IExecutionContext.

Usage considerations

Allowed context for the API call
- Thread-safe: Yes; if createExecutionContext fails, users should treat this as a critical error and not perform any subsequent TensorRT operations apart from outputting the error logs.

◆ createExecutionContextWithoutDeviceMemory()

virtual IExecutionContext * nvinfer1::safe::ICudaEngine::createExecutionContextWithoutDeviceMemory ( )

pure virtualnoexcept

Create an execution context without any device memory allocated.

The memory for execution of this device context must be supplied by the application.

See also: getDeviceMemorySize() safe::IExecutionContext::setDeviceMemory()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes; if createExecutionContext fails, users should treat this as a critical error and not perform any subsequent TensorRT operations apart from outputting the error logs.

◆ getBindingBytesPerComponent()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getBindingBytesPerComponent ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Return the number of bytes per component of an element.

The vector component size is returned if getBindingVectorizedDim() != -1.

Parameters

bindingIndex The binding Index.

See also: safe::ICudaEngine::getBindingVectorizedDim()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingComponentsPerElement()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getBindingComponentsPerElement ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Return the number of components included in one element.

The number of elements in the vectors is returned if getBindingVectorizedDim() != -1.

Parameters

bindingIndex The binding Index.

See also: safe::ICudaEngine::getBindingVectorizedDim()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingDataType()

virtual DataType nvinfer1::safe::ICudaEngine::getBindingDataType ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Determine the required data type for a buffer from its binding index.

Parameters

bindingIndex The binding index.

Returns: The type of the data in the buffer.

See also: getBindingIndex()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingDimensions()

virtual Dims nvinfer1::safe::ICudaEngine::getBindingDimensions ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Get the dimensions of a binding.

Parameters

bindingIndex The binding index.

Returns: The dimensions of the binding if the index is in range, otherwise Dims()

See also: getBindingIndex()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingFormat()

virtual TensorFormat nvinfer1::safe::ICudaEngine::getBindingFormat ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Return the binding format.

Parameters

bindingIndex The binding Index.

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingIndex()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getBindingIndex ( AsciiChar const *const name ) const

pure virtualnoexcept

Retrieve the binding index for a named tensor.

safe::IExecutionContext::enqueueV2() requires an array of buffers. Engine bindings map from tensor names to indices in this array. Binding indices are assigned at engine build time, and take values in the range [0 ... n-1] where n is the total number of inputs and outputs.

Warning: Strings passed to the runtime must be 1024 characters or less including NULL terminator and must be NULL terminated.

Parameters

name	The tensor name.

Returns: The binding index for the named tensor, or -1 if the name is not found.

See also: getNbBindings()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingName()

virtual AsciiChar const * nvinfer1::safe::ICudaEngine::getBindingName ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Retrieve the name corresponding to a binding index.

This is the reverse mapping to that provided by getBindingIndex().

Parameters

bindingIndex The binding index.

Returns: The name corresponding to the index, or nullptr if the index is out of range.

See also: getBindingIndex()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getBindingVectorizedDim()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getBindingVectorizedDim ( std::int32_t const bindingIndex ) const

pure virtualnoexcept

Return the dimension index that the buffer is vectorized.

Specifically -1 is returned if scalars per vector is 1.

Parameters

bindingIndex The binding Index.

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getDeviceMemorySize()

virtual size_t nvinfer1::safe::ICudaEngine::getDeviceMemorySize ( ) const

pure virtualnoexcept

Return the amount of device memory required by an execution context.

See also: safe::IExecutionContext::setDeviceMemory()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getErrorRecorder()

virtual IErrorRecorder * nvinfer1::safe::ICudaEngine::getErrorRecorder ( ) const

pure virtualnoexcept

Get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error reporter has not been inherited from the IRuntime, and setErrorReporter() has not been called.

Returns: A pointer to the IErrorRecorder object that has been registered.

See also: setErrorRecorder()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getName()

virtual AsciiChar const * nvinfer1::safe::ICudaEngine::getName ( ) const

pure virtualnoexcept

Returns the name of the network associated with the engine.

The name is set during network creation and is retrieved after building or deserialization.

See also: INetworkDefinition::setName(), INetworkDefinition::getName()

Returns: A null-terminated C-style string representing the name of the network.

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ getNbBindings()

virtual std::int32_t nvinfer1::safe::ICudaEngine::getNbBindings ( ) const

pure virtualnoexcept

Get the number of binding indices.

See also: getBindingIndex()

Usage considerations

Allowed context for the API call
- Thread-safe: Yes

◆ operator=() [1/2]

ICudaEngine & nvinfer1::safe::ICudaEngine::operator= ( ICudaEngine && ) &

delete

◆ operator=() [2/2]

ICudaEngine & nvinfer1::safe::ICudaEngine::operator= ( ICudaEngine const & ) &

delete

◆ setErrorRecorder()

virtual void nvinfer1::safe::ICudaEngine::setErrorRecorder ( IErrorRecorder *const recorder )

pure virtualnoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

Parameters

recorder The error recorder to register with this interface.

See also: getErrorRecorder()

Usage considerations

Allowed context for the API call
- Thread-safe: No

The documentation for this class was generated from the following file:

NvInferSafeRuntime.h

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ ICudaEngine() [1/3]

◆ ~ICudaEngine()

◆ ICudaEngine() [2/3]

◆ ICudaEngine() [3/3]

Member Function Documentation

◆ bindingIsInput()

◆ createExecutionContext()

◆ createExecutionContextWithoutDeviceMemory()

◆ getBindingBytesPerComponent()

◆ getBindingComponentsPerElement()

◆ getBindingDataType()

◆ getBindingDimensions()

◆ getBindingFormat()

◆ getBindingIndex()

◆ getBindingName()

◆ getBindingVectorizedDim()

◆ getDeviceMemorySize()

◆ getErrorRecorder()

◆ getName()

◆ getNbBindings()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ setErrorRecorder()