IPluginV2Ext

class tensorrt.IPluginV2Ext

Plugin class for user-implemented layers.

Plugins are a mechanism for applications to implement custom layers. This interface provides additional capabilities to the IPluginV2 interface by supporting different output data types.

Variables:tensorrt_version – int The API version with which this plugin was built.
attach_to_context(self: tensorrt.tensorrt.IPluginV2Ext, cudnn: capsule, cublas: capsule, allocator: capsule) → None

Attach the plugin object to an execution context and grant the plugin the access to some context resource.

:arg cudnn The cudnn context handle of the execution context :arg cublas The cublas context handle of the execution context :arg allocator The allocator used by the execution context

This function is called automatically for each plugin when a new execution context is created. If the plugin needs per-context resource, it can be allocated here. The plugin can also get context-owned CUDNN and CUBLAS context here.

clone(self: tensorrt.tensorrt.IPluginV2Ext) → tensorrt.tensorrt.IPluginV2Ext

Clone the plugin object. This copies over internal plugin parameters as well and returns a new plugin object with these parameters.

If the source plugin is pre-configured with configure_plugin(), the returned object should also be pre-configured. The returned object should allow attach_to_context() with a new execution context. Cloned plugin objects can share the same per-engine immutable resource (e.g. weights) with the source object (e.g. via ref-counting) to avoid duplication.

configure_plugin(self: tensorrt.tensorrt.IPluginV2Ext, input_shapes: List[tensorrt.tensorrt.Dims], output_shapes: List[tensorrt.tensorrt.Dims], input_types: List[tensorrt.tensorrt.DataType], output_types: List[tensorrt.tensorrt.DataType], input_is_broadcasted: List[bool], output_is_broacasted: List[bool], format: nvinfer1::TensorFormat, max_batch_size: int) → None

Configure the layer.

This function is called by the Builder prior to initialize() . It provides an opportunity for the layer to make algorithm choices on the basis of its weights, dimensions, and maximum batch size.

The dimensions passed here do not include the outermost batch size (i.e. for 2D image networks, they will be 3D CHW dimensions).

Parameters:
  • input_shapes – The shapes of the input tensors.
  • output_shapes – The shapes of the output tensors.
  • input_types – The data types of the input tensors.
  • output_types – The data types of the output tensors.
  • input_is_broadcasted – Whether an input is broadcasted across the batch.
  • output_is_broadcasted – Whether an output is broadcasted across the batch.
  • format – The format selected for floating-point inputs and outputs of the engine.
  • max_batch_size – The maximum batch size.
detach_from_context(self: tensorrt.tensorrt.IPluginV2Ext) → None

Detach the plugin object from its execution context.

This function is called automatically for each plugin when a execution context is destroyed. If the plugin owns per-context resource, it can be released here.

get_output_data_type(self: tensorrt.tensorrt.IPluginV2Ext, index: int, input_types: List[tensorrt.tensorrt.DataType]) → tensorrt.tensorrt.DataType

Return the DataType of the plugin output at the requested index. The default behavior should be to return the type of the first input, or DataType::kFLOAT if the layer has no inputs. The returned data type must have a format that is supported by the plugin.

Parameters:
  • index – Index of the output for which Data type is requested.
  • input_types – Data types of the inputs.
Returns:

DataType of the plugin output at the requested index.