An engine for executing inference on a built network. More...

#include <NvInfer.h>

Public Member Functions
virtual int	getNbBindings () const =0
	Get the number of binding indices. More...

virtual int	getBindingIndex (const char *name) const =0
	Retrieve the binding index for a named tensor. More...

virtual const char *	getBindingName (int bindingIndex) const =0
	Retrieve the name corresponding to a binding index. More...

virtual bool	bindingIsInput (int bindingIndex) const =0
	Determine whether a binding is an input binding. More...

virtual Dims	getBindingDimensions (int bindingIndex) const =0
	Get the dimensions of a binding. More...

virtual DataType	getBindingDataType (int bindingIndex) const =0
	Determine the required data type for a buffer from its binding index. More...

virtual int	getMaxBatchSize () const =0
	Get the maximum batch size which can be used for inference. More...

virtual int	getNbLayers () const =0
	Get the number of layers in the network. More...

virtual std::size_t	getWorkspaceSize () const =0
	Get the amount of workspace the engine uses. More...

virtual IHostMemory *	serialize () const =0
	Serialize the network to a stream. More...

virtual IExecutionContext *	createExecutionContext ()=0
	Create an execution context. More...

virtual void	destroy ()=0
	Destroy this object;.

virtual TensorLocation	getLocation (int bindingIndex) const =0
	Get location of binding. More...

virtual IExecutionContext *	createExecutionContextWithoutDeviceMemory ()=0
	create an execution context without any device memory allocated More...

virtual size_t	getDeviceMemorySize () const =0
	Return the amount of device memory required by an execution context. More...

virtual bool	isRefittable () const =0
	Return true if engine can be refit. More...

Detailed Description

An engine for executing inference on a built network.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

virtual bool nvinfer1::ICudaEngine::bindingIsInput ( int bindingIndex ) const

pure virtual

Determine whether a binding is an input binding.

Parameters

bindingIndex The binding index.

Returns: True if the index corresponds to an input binding and the index is in range.

See Also: getBindingIndex()

virtual IExecutionContext* nvinfer1::ICudaEngine::createExecutionContext ( )

pure virtual

Create an execution context.

See Also: IExecutionContext.

virtual IExecutionContext* nvinfer1::ICudaEngine::createExecutionContextWithoutDeviceMemory ( )

pure virtual

create an execution context without any device memory allocated

The memory for execution of this device context must be supplied by the application.

See Also: getDeviceMemorySize() IExecutionContext::setDeviceMemory()

virtual DataType nvinfer1::ICudaEngine::getBindingDataType ( int bindingIndex ) const

pure virtual

Determine the required data type for a buffer from its binding index.

Parameters

bindingIndex The binding index.

Returns: The type of the data in the buffer.

See Also: getBindingIndex()

virtual Dims nvinfer1::ICudaEngine::getBindingDimensions ( int bindingIndex ) const

pure virtual

Get the dimensions of a binding.

Parameters

bindingIndex The binding index.

Returns: The dimensions of the binding if the index is in range, otherwise (0,0,0).

See Also: getBindingIndex()

virtual int nvinfer1::ICudaEngine::getBindingIndex ( const char * name ) const

pure virtual

Retrieve the binding index for a named tensor.

IExecutionContext::enqueue() and IExecutionContext::execute() require an array of buffers.

Engine bindings map from tensor names to indices in this array. Binding indices are assigned at engine build time, and take values in the range [0 ... n-1] where n is the total number of inputs and outputs.

Parameters

name	The tensor name.

Returns: The binding index for the named tensor, or -1 if the name is not found.

see getNbBindings() getBindingIndex()

virtual const char* nvinfer1::ICudaEngine::getBindingName ( int bindingIndex ) const

pure virtual

Retrieve the name corresponding to a binding index.

This is the reverse mapping to that provided by getBindingIndex().

Parameters

bindingIndex The binding index.

Returns: The name corresponding to the index, or nullptr if the index is out of range.

See Also: getBindingIndex()

virtual size_t nvinfer1::ICudaEngine::getDeviceMemorySize ( ) const

pure virtual

Return the amount of device memory required by an execution context.

See Also: IExecutionContext::setDeviceMemory()

virtual TensorLocation nvinfer1::ICudaEngine::getLocation ( int bindingIndex ) const

pure virtual

Get location of binding.

This lets you know whether the binding should be a pointer to device or host memory.

See Also: ITensor::setLocation() ITensor::getLocation()

Parameters

bindingIndex The binding index.

Returns: The location of the bound tensor with given index.

virtual int nvinfer1::ICudaEngine::getMaxBatchSize ( ) const

pure virtual

Get the maximum batch size which can be used for inference.

Returns: The maximum batch size for this engine.

virtual int nvinfer1::ICudaEngine::getNbBindings ( ) const

pure virtual

Get the number of binding indices.

See Also: getBindingIndex();

virtual int nvinfer1::ICudaEngine::getNbLayers ( ) const

pure virtual

Get the number of layers in the network.

The number of layers in the network is not necessarily the number in the original network definition, as layers may be combined or eliminated as the engine is optimized. This value can be useful when building per-layer tables, such as when aggregating profiling data over a number of executions.

Returns: The number of layers in the network.

virtual std::size_t nvinfer1::ICudaEngine::getWorkspaceSize ( ) const

pure virtual

Get the amount of workspace the engine uses.

The workspace size will be no greater than the value provided to the builder when the engine was built, and will typically be smaller. Workspace will be allocated for each execution context.

virtual bool nvinfer1::ICudaEngine::isRefittable ( ) const

pure virtual

Return true if engine can be refit.

See Also: nvinfer1::createInferRefitter()

virtual IHostMemory* nvinfer1::ICudaEngine::serialize ( ) const

pure virtual

Serialize the network to a stream.

Returns: A IHostMemory object that contains the serialized engine.

The network may be deserialized with IRuntime::deserializeCudaEngine()

See Also: IRuntime::deserializeCudaEngine()

The documentation for this class was generated from the following file:

NvInfer.h

Public Member Functions

Detailed Description

Member Function Documentation