ICudaEngine

class tensorrt.ICudaEngine

An ICudaEngine for executing inference on a built network.

The engine can be indexed with [] . When indexed in this way with an integer, it will return the corresponding binding name. When indexed with a string, it will return the corresponding binding index.

Variables:
  • num_bindings – int The number of binding indices.
  • max_batch_size – int The maximum batch size which can be used for inference.
  • num_layers – int The number of layers in the network. The number of layers in the network is not necessarily the number in the original INetworkDefinition, as layers may be combined or eliminated as the ICudaEngine is optimized. This value can be useful when building per-layer tables, such as when aggregating profiling data over a number of executions.
  • max_workspace_size – int The amount of workspace the ICudaEngine uses. The workspace size will be no greater than the value provided to the Builder when the ICudaEngine was built, and will typically be smaller. Workspace will be allocated for each IExecutionContext .
  • device_memory_size – int The amount of device memory required by an IExecutionContext .
binding_is_input(*args, **kwargs)

Overloaded function.

  1. binding_is_input(self: tensorrt.tensorrt.ICudaEngine, index: int) -> bool

    Determine whether a binding is an input binding.

    index:The binding index.
    returns:True if the index corresponds to an input binding and the index is in range.

  2. binding_is_input(self: tensorrt.tensorrt.ICudaEngine, name: str) -> bool

    Determine whether a binding is an input binding.

    name:The name of the tensor corresponding to an engine binding.
    returns:True if the index corresponds to an input binding and the index is in range.
create_execution_context(self: tensorrt.tensorrt.ICudaEngine) → tensorrt.tensorrt.IExecutionContext

Create an IExecutionContext .

Returns:The newly created IExecutionContext .
create_execution_context_without_device_memory(self: tensorrt.tensorrt.ICudaEngine) → tensorrt.tensorrt.IExecutionContext

Create an IExecutionContext without any device memory allocated The memory for execution of this device context must be supplied by the application.

Returns:An IExecutionContext without device memory allocated.
get_binding_dtype(*args, **kwargs)

Overloaded function.

  1. get_binding_dtype(self: tensorrt.tensorrt.ICudaEngine, index: int) -> tensorrt.tensorrt.DataType

    Determine the required data type for a buffer from its binding index.

    index:The binding index.
    Returns:The type of data in the buffer.

  2. get_binding_dtype(self: tensorrt.tensorrt.ICudaEngine, name: str) -> tensorrt.tensorrt.DataType

    Determine the required data type for a buffer from its binding index.

    name:The name of the tensor corresponding to an engine binding.
    Returns:The type of data in the buffer.
get_binding_index(self: tensorrt.tensorrt.ICudaEngine, name: str) → int

Retrieve the binding index for a named tensor.

You can also use engine’s __getitem__() with engine[name]. When invoked with a str , this will return the corresponding binding index.

IExecutionContext.execute_async() and IExecutionContext.execute() require an array of buffers. Engine bindings map from tensor names to indices in this array. Binding indices are assigned at ICudaEngine build time, and take values in the range [0 … n-1] where n is the total number of inputs and outputs.

Parameters:name – The tensor name.
Returns:The binding index for the named tensor, or -1 if the name is not found.
get_binding_name(self: tensorrt.tensorrt.ICudaEngine, index: int) → str

Retrieve the name corresponding to a binding index.

You can also use engine’s __getitem__() with engine[index]. When invoked with an int , this will return the corresponding binding name.

This is the reverse mapping to that provided by get_binding_index() .

Parameters:index – The binding index.
Returns:The name corresponding to the binding index.
get_binding_shape(*args, **kwargs)

Overloaded function.

  1. get_binding_shape(self: tensorrt.tensorrt.ICudaEngine, index: int) -> tensorrt.tensorrt.Dims

    Get the shape of a binding.

    index:The binding index.
    Returns:The shape of the binding if the index is in range, otherwise (0, 0, 0).

  2. get_binding_shape(self: tensorrt.tensorrt.ICudaEngine, name: str) -> tensorrt.tensorrt.Dims

    Get the shape of a binding.

    name:The name of the tensor corresponding to an engine binding.
    Returns:The shape of the binding if the tensor is present, otherwise (0, 0, 0).
get_location(*args, **kwargs)

Overloaded function.

  1. get_location(self: tensorrt.tensorrt.ICudaEngine, index: int) -> tensorrt.tensorrt.TensorLocation

    Get location of binding. This lets you know whether the binding should be a pointer to device or host memory.

    index:The binding index.
    returns:The location of the bound tensor with given index.

  2. get_location(self: tensorrt.tensorrt.ICudaEngine, name: str) -> tensorrt.tensorrt.TensorLocation

    Get location of binding. This lets you know whether the binding should be a pointer to device or host memory.

    name:The name of the tensor corresponding to an engine binding.
    returns:The location of the bound tensor with given index.
serialize(self: tensorrt.tensorrt.ICudaEngine) → tensorrt.tensorrt.IHostMemory

Serialize the network to a stream.

Returns:An IHostMemory object containing the serialized ICudaEngine .