Layer Base Classes¶

ITensor¶

tensorrt.TensorLocation

The physical location of the data.

Members:

DEVICE : Data is stored on the device.

HOST : Data is stored on the device.

class tensorrt.ITensor

A tensor in an INetworkDefinition .

Variables: name – str The tensor name. For a network input, the name is assigned by the application. For tensors which are layer outputs, a default name is assigned consisting of the layer name followed by the index of the output in brackets. shape – Dims The shape of a tensor. For a network input the shape is assigned by the application. For a network output it is computed based on the layer parameters and the inputs to the layer. If a tensor size or a parameter is modified in the network, the shape of all dependent tensors will be recomputed. This call is only legal for network input tensors, since the shape of layer output tensors are inferred based on layer inputs and parameters. dtype – DataType The data type of a tensor. The type is unchanged if the type is invalid for the given tensor. If the tensor is a network input or output, then the tensor type cannot be tensorrt.int8 . broadcast_across_batch – bool Whether to enable broadcast of tensor across the batch. When a tensor is broadcast across a batch, it has the same value for every member in the batch. Memory is only allocated once for the single member. This method is only valid for network input tensors, since the flags of layer output tensors are inferred based on layer inputs and parameters. If this state is modified for a tensor in the network, the states of all dependent tensors will be recomputed. location – TensorLocation The storage location of a tensor. is_network_input – bool Whether the tensor is a network input. is_network_output – bool Whether the tensor is a network output. dynamic_range – Tuple[float, float] A tuple containing the [minimum, maximum] of the dynamic range, or None if the range was not set.
get_dynamic_range(self: tensorrt.tensorrt.ITensor) → float

Get dynamic range for the tensor. NOTE: It is suggested to use tensor.dynamic_range instead, which is a tuple including both the minimum and maximum of the dynamic range.

Returns: The absolute maximum of the dynamic range.
reset_dynamic_range(self: tensorrt.tensorrt.ITensor) → None

Undo the effect of setting the dynamic range.

set_dynamic_range(self: tensorrt.tensorrt.ITensor, min: float, max: float) → bool

Set dynamic range for the tensor. NOTE: It is suggested to use tensor.dynamic_range = (min, max) instead.

Parameters: min – Minimum of the dynamic range. max – Maximum of the dyanmic range. true if succeed in setting range. Otherwise false.

ILayer¶

tensorrt.LayerType

Type of Layer

Members:

CONCATENATION : Concatenation layer

TOPK : TopK layer

CONVOLUTION : Convolution layer

RNN : RNN layer

UNARY : Unary layer

ACTIVATION : Activation layer

IDENTITY : Identity layer

CONSTANT : Constant layer

DECONVOLUTION : Deconvolution layer

RNN_V2 : RNNv2 layer

ELEMENTWISE : Elementwise layer

POOLING : Pooling layer

PLUGIN_V2 : PluginV2 layer

FULLY_CONNECTED : Fully connected layer

SCALE : Scale layer

SOFTMAX : Softmax layer

GATHER : Gather layer

SLICE : Slice layer

REDUCE : Reduce layer

LRN : LRN layer

SHUFFLE : Shuffle layer

RAGGED_SOFTMAX : Ragged softmax layer

MATRIX_MULTIPLY : Matrix multiply layer

PLUGIN : Plugin layer

class tensorrt.ILayer

Base class for all layer classes in an INetworkDefinition .

Variables: name – str The name of the layer. type – LayerType The type of the layer. num_inputs – int The number of inputs of the layer. num_outputs – int The number of outputs of the layer. precision – DataType The computation precision. precision_is_set – bool Whether the precision is set or not.
get_input(self: tensorrt.tensorrt.ILayer, index: int) → tensorrt.tensorrt.ITensor

Get the layer input corresponding to the given index.

Parameters: index – The index of the input tensor. The input tensor, or None if the index is out of range.
get_output(self: tensorrt.tensorrt.ILayer, index: int) → tensorrt.tensorrt.ITensor

Get the layer output corresponding to the given index.

Parameters: index – The index of the output tensor. The output tensor, or None if the index is out of range.
get_output_type(self: tensorrt.tensorrt.ILayer, index: int) → tensorrt.tensorrt.DataType

Get the output type of the layer.

Parameters: index – The index of the output tensor. The output precision. Default : DataType.FLOAT.
output_type_is_set(self: tensorrt.tensorrt.ILayer, index: int) → bool

Whether the output type has been set for this layer.

Parameters: index – The index of the output. Whether the output type has been explicitly set.
reset_output_type(self: tensorrt.tensorrt.ILayer, index: int) → None

Reset output type of this layer.

Parameters: index – The index of the output.
reset_precision(self: tensorrt.tensorrt.ILayer) → None

Reset the computation precision of the layer.

set_output_type(self: tensorrt.tensorrt.ILayer, index: int, dtype: tensorrt.tensorrt.DataType) → None

Constraint layer to generate output data with given type. Note that this method cannot be used to set the data type of the second output tensor of the topK layer. The data type of the second output tensor of the topK layer is always Int32.

Parameters: index – The index of the output tensor to set the type. dtype – DataType of the output.

IOutputDimensionsFormula¶

class tensorrt.IOutputDimensionsFormula

Application-implemented interface to compute layer output sizes.

compute(self: tensorrt.tensorrt.IOutputDimensionsFormula, input_shape: tensorrt.tensorrt.DimsHW, kernel_shape: tensorrt.tensorrt.DimsHW, stride: tensorrt.tensorrt.DimsHW, padding: tensorrt.tensorrt.DimsHW, dilation: tensorrt.tensorrt.DimsHW, layer_name: str) → tensorrt.tensorrt.DimsHW

Application-implemented interface to compute the HW output dimensions of a layer from the layer input and parameters.

Parameters: input_shape – The input shape of the layer. kernel_shape – The kernel shape (or window size, for a pooling layer) parameter of the layer operation. stride – The stride parameter for the layer. padding – The padding parameter of the layer. dilation – The dilation parameter of the layer (only applicable to convolutions). layer_name – The name of the layer. The output size of the layer