Context for executing inference using an engine. More...

#include <NvInfer.h>

Public Member Functions
virtual bool	execute (int batchSize, void **bindings)=0
	Synchronously execute inference on a batch. More...

virtual bool	enqueue (int batchSize, void *bindings, cudaStream_t stream, cudaEvent_t inputConsumed)=0
	Asynchronously execute inference on a batch. More...

virtual void	setDebugSync (bool sync)=0
	Set the debug sync flag. More...

virtual bool	getDebugSync () const =0
	Get the debug sync flag. More...

virtual void	setProfiler (IProfiler *)=0
	Set the profiler. More...

virtual IProfiler *	getProfiler () const =0
	Get the profiler. More...

virtual const ICudaEngine &	getEngine () const =0
	Get the associated engine. More...

virtual void	destroy ()=0
	Destroy this object.

virtual void	setName (const char *name)=0
	Set the name of the execution context. More...

virtual const char *	getName () const =0
	Return the name of the execution context. More...

virtual void	setDeviceMemory (void *memory)=0
	set the device memory for use by this execution context. More...

Detailed Description

Context for executing inference using an engine.

Multiple execution contexts may exist for one ICudaEngine instance, allowing the same engine to be used for the execution of multiple batches simultaneously.

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

virtual bool nvinfer1::IExecutionContext::enqueue	(	int	batchSize,
		void **	bindings,
		cudaStream_t	stream,
		cudaEvent_t *	inputConsumed
	)

pure virtual

Asynchronously execute inference on a batch.

This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex()

Parameters

batchSize	The batch size. This is at most the value supplied when the engine was built.
bindings	An array of pointers to input and output buffers for the network.
stream	A cuda stream on which the inference kernels will be enqueued
inputConsumed	An optional event which will be signaled when the input buffers can be refilled with new data

Returns: True if the kernels were enqueued successfully.

See Also: ICudaEngine::getBindingIndex() ICudaEngine::getMaxBatchSize()

virtual bool nvinfer1::IExecutionContext::execute	(	int	batchSize,
		void **	bindings
	)

pure virtual

Synchronously execute inference on a batch.

This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex()

Parameters

batchSize	The batch size. This is at most the value supplied when the engine was built.
bindings	An array of pointers to input and output buffers for the network.

Returns: True if execution succeeded.

See Also: ICudaEngine::getBindingIndex() ICudaEngine::getMaxBatchSize()

virtual bool nvinfer1::IExecutionContext::getDebugSync ( ) const

pure virtual

Get the debug sync flag.

See Also: setDebugSync()

virtual const ICudaEngine& nvinfer1::IExecutionContext::getEngine ( ) const

pure virtual

Get the associated engine.

See Also: ICudaEngine

virtual const char* nvinfer1::IExecutionContext::getName ( ) const

pure virtual

Return the name of the execution context.

See Also: setName()

virtual IProfiler* nvinfer1::IExecutionContext::getProfiler ( ) const

pure virtual

Get the profiler.

See Also: IProfiler setProfiler()

virtual void nvinfer1::IExecutionContext::setDebugSync ( bool sync )

pure virtual

Set the debug sync flag.

If this flag is set to true, the engine will log the successful execution for each kernel during execute(). It has no effect when using enqueue().

See Also: getDebugSync()

virtual void nvinfer1::IExecutionContext::setDeviceMemory ( void * memory )

pure virtual

set the device memory for use by this execution context.

The memory must be aligned with cuda memory alignment property (using cudaGetDeviceProperties()), and its size must be at least that returned by getDeviceMemorySize(). If using enqueue() to run the network, The memory is in use from the invocation of enqueue() until network execution is complete. If using execute(), it is in use until execute() returns. Releasing or otherwise using the memory for other purposes during this time will result in undefined behavior.

See Also: ICudaEngine::getDeviceMemorySize() ICudaEngine::createExecutionContextWithoutDeviceMemory()

virtual void nvinfer1::IExecutionContext::setName ( const char * name )

pure virtual

Set the name of the execution context.

This method copies the name string.

See Also: getName()

virtual void nvinfer1::IExecutionContext::setProfiler ( IProfiler * )

pure virtual

Set the profiler.

See Also: IProfiler getProfiler()

The documentation for this class was generated from the following file:

NvInfer.h

Public Member Functions

Detailed Description

Member Function Documentation