nvinfer1 Namespace Reference

The TensorRT API version 1 namespace. More...

Classes
class	Dims
	Structure to define the dimensions of a tensor. More...
class	Dims2
	Descriptor for two-dimensional data. More...
class	DimsHW
	Descriptor for two-dimensional spatial data. More...
class	Dims3
	Descriptor for three-dimensional data. More...
class	DimsCHW
	Descriptor for data with one channel dimension and two spatial dimensions. More...
class	Dims4
	Descriptor for four-dimensional data. More...
class	DimsNCHW
	Descriptor for data with one index dimension, one channel dimension and two spatial dimensions. More...
class	Weights
	An array of weights used as a layer parameter. More...
class	IHostMemory
	Class to handle library allocated memory that is accessible to the user. More...
class	ITensor
	A tensor in a network definition. More...
class	ILayer
	Base class for all layer classes in a network definition. More...
class	IConvolutionLayer
	A convolution layer in a network definition. More...
class	IFullyConnectedLayer
	A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...
class	IActivationLayer
	An Activation layer in a network definition. More...
class	IPoolingLayer
	A Pooling layer in a network definition. More...
class	ILRNLayer
	A LRN layer in a network definition. More...
class	IScaleLayer
	A Scale layer in a network definition. More...
class	ISoftMaxLayer
	A Softmax layer in a network definition. More...
class	IConcatenationLayer
	A concatenation layer in a network definition. More...
class	IDeconvolutionLayer
	A deconvolution layer in a network definition. More...
class	IElementWiseLayer
	A elementwise layer in a network definition. More...
class	IGatherLayer
class	IRNNLayer
	A RNN layer in a network definition. More...
class	IRNNv2Layer
	An RNN layer in a network definition, version 2. More...
class	IOutputDimensionsFormula
	Application-implemented interface to compute layer output sizes. More...
class	IPlugin
	Plugin class for user-implemented layers. More...
class	IPluginExt
	Plugin class for user-implemented layers. More...
class	IPluginV2
	Plugin class for user-implemented layers. More...
class	IPluginV2Ext
	Plugin class for user-implemented layers. More...
class	IPluginLayer
	Layer type for plugins. More...
class	IPluginV2Layer
	Layer type for pluginV2. More...
class	PluginField
	Structure containing plugin attribute field names and associated data This information can be parsed to decode necessary plugin metadata. More...
struct	PluginFieldCollection
class	IPluginCreator
	Plugin creator class for user implemented layers. More...
class	IPluginRegistry
	Single registration point for all plugins in an application. It is used to find plugin implementations during engine deserialization. Internally, the plugin registry is considered to be a singleton so all plugins in an application are part of the same global registry. Note that the plugin registry is only supported for plugins of type IPluginV2 and should also have a corresponding IPluginCreator implementation. More...
class	IUnaryLayer
	Layer that represents an unary operation. More...
class	IReduceLayer
	Layer that represents a reduction operator. More...
class	IPaddingLayer
	Layer that represents a padding operation. More...
struct	Permutation
class	IShuffleLayer
	Layer type for shuffling data. More...
class	ISliceLayer
class	ITopKLayer
	Layer that represents a TopK reduction. More...
class	IMatrixMultiplyLayer
	Layer that represents a Matrix Multiplication. More...
class	IRaggedSoftMaxLayer
	A RaggedSoftmax layer in a network definition. More...
class	IIdentityLayer
	A layer that represents the identity function. More...
class	IConstantLayer
	Layer that represents a constant value. More...
class	INetworkDefinition
	A network definition for input to the builder. More...
class	IProfiler
	Application-implemented interface for profiling. More...
class	IExecutionContext
	Context for executing inference using an engine. More...
class	ICudaEngine
	An engine for executing inference on a built network. More...
class	IInt8Calibrator
	Application-implemented interface for calibration. More...
class	IInt8EntropyCalibrator
class	IInt8EntropyCalibrator2
class	IInt8LegacyCalibrator
class	IGpuAllocator
	Application-implemented class for controlling allocation on the GPU. More...
class	IBuilder
	Builds an engine from a network definition. More...
class	IRefitter
	Updates weights in an engine. More...
class	IPluginFactory
	Plugin factory for deserialization. More...
class	IRuntime
	Allows a serialized engine to be deserialized. More...
class	ILogger
	Application-implemented logging interface for the builder, engine and runtime. More...
class	PluginRegistrar
	Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded. This static object will register all creators available in the library to the registry. More...

Enumerations
enum	DataType : int { DataType::kFLOAT = 0, DataType::kHALF = 1, DataType::kINT8 = 2, DataType::kINT32 = 3 }
	The type of weights and tensors. More...
enum	DeviceType : int { DeviceType::kGPU, DeviceType::kDLA }
	The device that this layer/network will execute on. More...
enum	DimensionType : int { DimensionType::kSPATIAL = 0, DimensionType::kCHANNEL = 1, DimensionType::kINDEX = 2, DimensionType::kSEQUENCE = 3 }
	The type of data encoded across this dimension. More...
enum	LayerType : int {   LayerType::kCONVOLUTION = 0, LayerType::kFULLY_CONNECTED = 1, LayerType::kACTIVATION = 2, LayerType::kPOOLING = 3,   LayerType::kLRN = 4, LayerType::kSCALE = 5, LayerType::kSOFTMAX = 6, LayerType::kDECONVOLUTION = 7,   LayerType::kCONCATENATION = 8, LayerType::kELEMENTWISE = 9, LayerType::kPLUGIN = 10, LayerType::kRNN = 11,   LayerType::kUNARY = 12, LayerType::kPADDING = 13, LayerType::kSHUFFLE = 14, LayerType::kREDUCE = 15,   LayerType::kTOPK = 16, LayerType::kGATHER = 17, LayerType::kMATRIX_MULTIPLY = 18, LayerType::kRAGGED_SOFTMAX = 19,   LayerType::kCONSTANT = 20, LayerType::kRNN_V2 = 21, LayerType::kIDENTITY = 22, LayerType::kPLUGIN_V2 = 23,   LayerType::kSLICE = 24 }
	The type values of layer classes. More...
enum	TensorLocation : int { TensorLocation::kDEVICE = 0, TensorLocation::kHOST = 1 }
	The location for tensor data storage, device or host. More...
enum	PaddingMode : int {   PaddingMode::kEXPLICIT_ROUND_DOWN = 0, PaddingMode::kEXPLICIT_ROUND_UP = 1, PaddingMode::kSAME_UPPER = 2, PaddingMode::kSAME_LOWER = 3,   PaddingMode::kCAFFE_ROUND_DOWN = 4, PaddingMode::kCAFFE_ROUND_UP = 5 }
	Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode gets precedence if setPaddingMode() and setPrePadding() are also used. More...
enum	ActivationType : int {   ActivationType::kRELU = 0, ActivationType::kSIGMOID = 1, ActivationType::kTANH = 2, ActivationType::kLEAKY_RELU = 3,   ActivationType::kELU = 4, ActivationType::kSELU = 5, ActivationType::kSOFTSIGN = 6, ActivationType::kSOFTPLUS = 7,   ActivationType::kCLIP = 8, ActivationType::kHARD_SIGMOID = 9, ActivationType::kSCALED_TANH = 10, ActivationType::kTHRESHOLDED_RELU = 11 }
	Enumerates the types of activation to perform in an activation layer. More...
enum	PoolingType : int { kMAX = 0, kAVERAGE = 1, kMAX_AVERAGE_BLEND = 2 }
	The type of pooling to perform in a pooling layer.
enum	ScaleMode : int { ScaleMode::kUNIFORM = 0, ScaleMode::kCHANNEL = 1, ScaleMode::kELEMENTWISE = 2 }
	Controls how shift, scale and power are applied in a Scale layer. More...
enum	ElementWiseOperation : int {   ElementWiseOperation::kSUM = 0, ElementWiseOperation::kPROD = 1, ElementWiseOperation::kMAX = 2, ElementWiseOperation::kMIN = 3,   ElementWiseOperation::kSUB = 4, ElementWiseOperation::kDIV = 5, ElementWiseOperation::kPOW = 6 }
	Enumerates the binary operations that may be performed by an ElementWise layer. More...
enum	RNNOperation : int { RNNOperation::kRELU = 0, RNNOperation::kTANH = 1, RNNOperation::kLSTM = 2, RNNOperation::kGRU = 3 }
	Enumerates the RNN operations that may be performed by an RNN layer. More...
enum	RNNDirection : int { RNNDirection::kUNIDIRECTION = 0, RNNDirection::kBIDIRECTION = 1 }
	Enumerates the RNN direction that may be performed by an RNN layer. More...
enum	RNNInputMode : int { RNNInputMode::kLINEAR = 0, RNNInputMode::kSKIP = 1 }
	Enumerates the RNN input modes that may occur with an RNN layer. More...
enum	RNNGateType : int {   RNNGateType::kINPUT = 0, RNNGateType::kOUTPUT = 1, RNNGateType::kFORGET = 2, RNNGateType::kUPDATE = 3,   RNNGateType::kRESET = 4, RNNGateType::kCELL = 5, RNNGateType::kHIDDEN = 6 }
	Identifies an individual gate within an RNN cell. More...
enum	PluginFormat : uint8_t { PluginFormat::kNCHW = 0, PluginFormat::kNC2HW2 = 1, PluginFormat::kNHWC8 = 2 }
enum	PluginFieldType : int {   PluginFieldType::kFLOAT16 = 0, PluginFieldType::kFLOAT32 = 1, PluginFieldType::kFLOAT64 = 2, PluginFieldType::kINT8 = 3,   PluginFieldType::kINT16 = 4, PluginFieldType::kINT32 = 5, PluginFieldType::kCHAR = 6, PluginFieldType::kDIMS = 7,   kUNKNOWN = 8 }
enum	UnaryOperation : int {   UnaryOperation::kEXP = 0, UnaryOperation::kLOG = 1, UnaryOperation::kSQRT = 2, UnaryOperation::kRECIP = 3,   UnaryOperation::kABS = 4, UnaryOperation::kNEG = 5, UnaryOperation::kSIN = 6, UnaryOperation::kCOS = 7,   UnaryOperation::kTAN = 8, UnaryOperation::kSINH = 9, UnaryOperation::kCOSH = 10, UnaryOperation::kASIN = 11,   UnaryOperation::kACOS = 12, UnaryOperation::kATAN = 13, UnaryOperation::kASINH = 14, UnaryOperation::kACOSH = 15,   UnaryOperation::kATANH = 16, UnaryOperation::kCEIL = 17, UnaryOperation::kFLOOR = 18 }
	Enumerates the unary operations that may be performed by a Unary layer. More...
enum	ReduceOperation : int {   kSUM = 0, kPROD = 1, kMAX = 2, kMIN = 3,   kAVG = 4 }
	Enumerates the reduce operations that may be performed by a Reduce layer.
enum	TopKOperation : int { TopKOperation::kMAX = 0, TopKOperation::kMIN = 1 }
	Enumerates the operations that may be performed by a TopK layer. More...
enum	MatrixOperation : int { MatrixOperation::kNONE, MatrixOperation::kTRANSPOSE, MatrixOperation::kVECTOR }
	Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication. More...
enum	CalibrationAlgoType : int { kLEGACY_CALIBRATION = 0, kENTROPY_CALIBRATION = 1, kENTROPY_CALIBRATION_2 = 2 }
	Version of calibration algorithm to use. More...
enum	EngineCapability { EngineCapability::kDEFAULT = 0, EngineCapability::kSAFE_GPU = 1, EngineCapability::kSAFE_DLA = 2 }
	List of supported engine capability flows. More...
enum	WeightsRole : int {   WeightsRole::kKERNEL = 0, WeightsRole::kBIAS = 1, WeightsRole::kSHIFT = 2, WeightsRole::kSCALE = 3,   WeightsRole::kCONSTANT = 4 }
	How a layer uses particular Weights. More...
enum	PluginType : int {   PluginType::kFASTERRCNN = 0, PluginType::kNORMALIZE = 1, PluginType::kPERMUTE = 2, PluginType::kPRIORBOX = 3,   PluginType::kSSDDETECTIONOUTPUT = 4, PluginType::kCONCAT = 5, PluginType::kPRELU = 6, PluginType::kYOLOREORG = 7,   PluginType::kYOLOREGION = 8, PluginType::kANCHORGENERATOR = 9 }
	The type values for the various plugins. More...

Functions
template<typename T >
int	EnumMax ()
	Maximum number of elements in an enumeration type.
template<>
int	EnumMax< DataType > ()
	Maximum number of elements in DataType enum. More...
template<>
int	EnumMax< DeviceType > ()
	Maximum number of elements in DeviceType enum. More...
template<>
int	EnumMax< DimensionType > ()
	Maximum number of elements in DimensionType enum. More...
template<>
int	EnumMax< LayerType > ()
	Maximum number of elements in LayerType enum. More...
template<>
int	EnumMax< TensorLocation > ()
	Maximum number of elements in TensorLocation enum. More...
template<>
int	EnumMax< PaddingMode > ()
	Maximum number of elements in PaddingMode enum. More...
template<>
int	EnumMax< ActivationType > ()
	Maximum number of elements in ActivationType enum. More...
template<>
int	EnumMax< PoolingType > ()
	Maximum number of elements in PoolingType enum. More...
template<>
int	EnumMax< ScaleMode > ()
	Maximum number of elements in ScaleMode enum. More...
template<>
int	EnumMax< ElementWiseOperation > ()
	Maximum number of elements in ElementWiseOperation enum. More...
template<>
int	EnumMax< RNNOperation > ()
	Maximum number of elements in RNNOperation enum. More...
template<>
int	EnumMax< RNNDirection > ()
	Maximum number of elements in RNNDirection enum. More...
template<>
int	EnumMax< RNNInputMode > ()
	Maximum number of elements in RNNInputMode enum. More...
template<>
int	EnumMax< RNNGateType > ()
	Maximum number of elements in RNNGateType enum. More...
template<>
int	EnumMax< PluginFormat > ()
	Maximum number of elements in PluginFormat enum. More...
template<>
int	EnumMax< UnaryOperation > ()
	Maximum number of elements in UnaryOperation enum. More...
template<>
int	EnumMax< ReduceOperation > ()
	Maximum number of elements in ReduceOperation enum. More...
template<>
int	EnumMax< TopKOperation > ()
	Maximum number of elements in TopKOperation enum. More...
template<>
int	EnumMax< MatrixOperation > ()
	Maximum number of elements in MatrixOperation enum. More...
template<>
int	EnumMax< CalibrationAlgoType > ()
	Maximum number of elements in CalibrationAlgoType enum. More...
template<>
int	EnumMax< EngineCapability > ()
	Maximum number of elements in EngineCapability enum. More...
template<>
int	EnumMax< WeightsRole > ()
	Maximum number of elements in WeightsRole enum. More...
template<>
int	EnumMax< ILogger::Severity > ()
	Maximum number of elements in ILogger::Severity enum. More...
template<>
int	EnumMax< PluginType > ()

Detailed Description

The TensorRT API version 1 namespace.

Enumeration Type Documentation

enum nvinfer1::ActivationType : int

strong

Enumerates the types of activation to perform in an activation layer.

Enumerator
kRELU	Rectified linear activation.
kSIGMOID	Sigmoid activation.
kTANH	TanH activation.
kLEAKY_RELU	LeakyRelu activation: x>=0 ? x : alpha * x.
kELU	Elu activation: x>=0 ? x : alpha * (exp(x) - 1).
kSELU	Selu activation: x>0 ? beta * x : beta * (alpha*exp(x) - alpha)
kSOFTSIGN	Softsign activation: x / (1+|x|)
kSOFTPLUS	Parametric softplus activation: alpha*log(exp(beta*x)+1)
kCLIP	Clip activation: max(alpha, min(beta, x))
kHARD_SIGMOID	Hard sigmoid activation: max(0, min(1, alpha*x+beta))
kSCALED_TANH	Scaled tanh activation: alpha*tanh(beta*x)
kTHRESHOLDED_RELU	Thresholded ReLU activation: x>alpha : x : 0.

enum nvinfer1::CalibrationAlgoType : int

strong

Version of calibration algorithm to use.

enum CalibrationAlgoType

enum nvinfer1::DataType : int

strong

The type of weights and tensors.

Enumerator
kFLOAT	FP32 format.
kHALF	FP16 format.
kINT8	quantized INT8 format.
kINT32	INT32 format.

enum nvinfer1::DeviceType : int

strong

The device that this layer/network will execute on.

Enumerator
kGPU	GPU Device.
kDLA	DLA Core.

enum nvinfer1::DimensionType : int

strong

The type of data encoded across this dimension.

Enumerator
kSPATIAL	Elements correspond to different spatial data.
kCHANNEL	Elements correspond to different channels.
kINDEX	Elements correspond to different batch index.
kSEQUENCE	Elements correspond to different sequence values.

enum nvinfer1::ElementWiseOperation : int

strong

Enumerates the binary operations that may be performed by an ElementWise layer.

See Also

IElementWiseLayer

Enumerator
kSUM	Sum of the two elements.
kPROD	Product of the two elements.
kMAX	Maximum of the two elements.
kMIN	Minimum of the two elements.
kSUB	Substract the second element from the first.
kDIV	Divide the first element by the second.
kPOW	The first element to the power of the second element.

enum nvinfer1::EngineCapability

strong

List of supported engine capability flows.

Note

at present, kSAFE_DLA flow doesn't strictly limit execution to DLA/PVA devices - it simply restricts the engine capabilities to DLA/PVA support levels anticipated in future releases.

Enumerator
kDEFAULT	Full capability, TensorRT mode without any restrictions.
kSAFE_GPU	Safety restricted capability, TensorRT flow that can only run on GPU devices.
kSAFE_DLA	Safety restricted capability, TensorRT flow that can only run on DLA/PVA devices.

enum nvinfer1::LayerType : int

strong

The type values of layer classes.

See Also

ILayer::getType()

Enumerator
kCONVOLUTION	Convolution layer.
kFULLY_CONNECTED	Fully connected layer.
kACTIVATION	Activation layer.
kPOOLING	Pooling layer.
kLRN	LRN layer.
kSCALE	Scale layer.
kSOFTMAX	SoftMax layer.
kDECONVOLUTION	Deconvolution layer.
kCONCATENATION	Concatenation layer.
kELEMENTWISE	Elementwise layer.
kPLUGIN	Plugin layer.
kRNN	RNN layer.
kUNARY	UnaryOp operation Layer.
kPADDING	Padding layer.
kSHUFFLE	Shuffle layer.
kREDUCE	Reduce layer.
kTOPK	TopK layer.
kGATHER	Gather layer.
kMATRIX_MULTIPLY	Matrix multiply layer.
kRAGGED_SOFTMAX	Ragged softmax layer.
kCONSTANT	Constant layer.
kRNN_V2	RNNv2 layer.
kIDENTITY	Identity layer.
kPLUGIN_V2	PluginV2 layer.
kSLICE	Slice layer.

enum nvinfer1::MatrixOperation : int

strong

Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication.

Enumerator
kNONE	Treat x as a matrix if it has two dimensions, or as a collection of matrices if x has more than two dimensions, where the last two dimensions are the matrix dimensions. x must have at least two dimensions.
kTRANSPOSE	Like kNONE, but transpose the matrix dimensions.
kVECTOR	Treat x as a vector if it has one dimension, or as a collection of vectors if x has more than one dimension. x must have at least one dimension.

enum nvinfer1::PaddingMode : int

strong

Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode gets precedence if setPaddingMode() and setPrePadding() are also used.

kEXPLICIT* padding is to use explicit padding. kSAME* padding is to implicitly calculate padding to keep output dim to be the "same" with input dim. For convolution and pooling, output dim is ceil(input dim, stride), for deconvolution it is inverse, then use the output dim to calculate padding size. kCAFFE* padding is symmetric padding.

Enumerator
kEXPLICIT_ROUND_DOWN	Use explicit padding, rounding output size down.
kEXPLICIT_ROUND_UP	Use explicit padding, rounding output size up.
kSAME_UPPER	Use SAME padding with prePadding <= postPadding.
kSAME_LOWER	Use SAME padding, with prePadding >= postPadding.
kCAFFE_ROUND_DOWN	Use CAFFE padding, rounding output size down.
kCAFFE_ROUND_UP	Use CAFFE padding, rounding output size up.

enum nvinfer1::PluginFieldType : int

strong

Enumerator
kFLOAT16	FP16 field type.
kFLOAT32	FP32 field type.
kFLOAT64	FP64 field type.
kINT8	INT8 field type.
kINT16	INT16 field type.
kINT32	INT32 field type.
kCHAR	char field type.
kDIMS	nvinfer1::Dims field type.

enum nvinfer1::PluginFormat : uint8_t

strong

Enumerator
kNCHW	NCHW.
kNC2HW2	NCHW with 2-element packed channels. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+1)/2][H][W][2], with the tensor coordinates (n,c,h,w) mapping to array subscript [n][c/2][h][w][c%2].
kNHWC8	NHWC where C must be a multiple of 8.

enum nvinfer1::PluginType : int

strong

The type values for the various plugins.

See Also

INvPlugin::getPluginType()

Enumerator
kFASTERRCNN	FasterRCNN fused plugin (RPN + ROI pooling).
kNORMALIZE	Normalize plugin.
kPERMUTE	Permute plugin.
kPRIORBOX	PriorBox plugin.
kSSDDETECTIONOUTPUT	SSD DetectionOutput plugin.
kCONCAT	Concat plugin.
kPRELU	YOLO PReLU Plugin.
kYOLOREORG	YOLO Reorg Plugin.
kYOLOREGION	YOLO Region Plugin.
kANCHORGENERATOR	SSD Grid Anchor Generator.

enum nvinfer1::RNNDirection : int

strong

Enumerates the RNN direction that may be performed by an RNN layer.

See Also

IRNNLayer, IRNNv2Layer

Enumerator
kUNIDIRECTION	Network iterations from first input to last input.
kBIDIRECTION	Network iterates from first to last and vice versa and outputs concatenated.

enum nvinfer1::RNNGateType : int

strong

Identifies an individual gate within an RNN cell.

See Also

RNNOperation

Enumerator
kINPUT	Input gate (i).
kOUTPUT	Output gate (o).
kFORGET	Forget gate (f).
kUPDATE	Update gate (z).
kRESET	Reset gate (r).
kCELL	Cell gate (c).
kHIDDEN	Hidden gate (h).

enum nvinfer1::RNNInputMode : int

strong

Enumerates the RNN input modes that may occur with an RNN layer.

If the RNN is configured with RNNInputMode::kLINEAR, then for each gate g in the first layer of the RNN, the input vector X[t] (length E) is left-multiplied by the gate's corresponding weight matrix W[g] (dimensions HxE) as usual, before being used to compute the gate output as described by RNNOperation.

If the RNN is configured with RNNInputMode::kSKIP, then this initial matrix multiplication is "skipped" and W[g] is conceptually an identity matrix. In this case, the input vector X[t] must have length H (the size of the hidden state).

See Also

IRNNLayer, IRNNv2Layer

Enumerator
kLINEAR	Perform the normal matrix multiplication in the first recurrent layer.
kSKIP	No operation is performed on the first recurrent layer.

enum nvinfer1::RNNOperation : int

strong

Enumerates the RNN operations that may be performed by an RNN layer.

Equation definitions

In the equations below, we use the following naming convention:

t := current time step

i := input gate
o := output gate
f := forget gate
z := update gate
r := reset gate
c := cell gate
h := hidden gate

g[t] denotes the output of gate g at timestep t, e.g.
f[t] is the output of the forget gate f.

X[t] := input tensor for timestep t
C[t] := cell state for timestep t
H[t] := hidden state for timestep t

W[g] := W (input) parameter weight matrix for gate g
R[g] := U (recurrent) parameter weight matrix for gate g
Wb[g] := W (input) parameter bias vector for gate g
Rb[g] := U (recurrent) parameter bias vector for gate g

Unless otherwise specified, all operations apply pointwise
to elements of each operand tensor.

ReLU(X) := max(X, 0)
tanh(X) := hyperbolic tangent of X
sigmoid(X) := 1 / (1 + exp(-X))
exp(X) := e^X

A.B denotes matrix multiplication of A and B.
A*B denotes pointwise multiplication of A and B.

Equations

Depending on the value of RNNOperation chosen, each sub-layer of the RNN layer will perform one of the following operations:

::kRELU

  H[t] := ReLU(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])

::kTANH

  H[t] := tanh(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])

::kLSTM

  i[t] := sigmoid(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
  f[t] := sigmoid(W[f].X[t] + R[f].H[t-1] + Wb[f] + Rb[f])
  o[t] := sigmoid(W[o].X[t] + R[o].H[t-1] + Wb[o] + Rb[o])
  c[t] :=    tanh(W[c].X[t] + R[c].H[t-1] + Wb[c] + Rb[c])

  C[t] := f[t]*C[t-1] + i[t]*c[t]
  H[t] := o[t]*tanh(C[t])

::kGRU

  z[t] := sigmoid(W[z].X[t] + R[z].H[t-1] + Wb[z] + Rb[z])
  r[t] := sigmoid(W[r].X[t] + R[r].H[t-1] + Wb[r] + Rb[r])
  h[t] := tanh(W[h].X[t] + r[t]*(R[h].H[t-1] + Rb[h]) + Wb[h])

  H[t] := (1 - z[t])*h[t] + z[t]*H[t-1]

See Also

IRNNLayer, IRNNv2Layer

Enumerator
kRELU	Single gate RNN w/ ReLU activation function.
kTANH	Single gate RNN w/ TANH activation function.
kLSTM	Four-gate LSTM network w/o peephole connections.
kGRU	Three-gate network consisting of Gated Recurrent Units.

enum nvinfer1::ScaleMode : int

strong

Controls how shift, scale and power are applied in a Scale layer.

See Also

IScaleLayer

Enumerator
kUNIFORM	Identical coefficients across all elements of the tensor.
kCHANNEL	Per-channel coefficients. The channel dimension is assumed to be the third to last dimension.
kELEMENTWISE	Elementwise coefficients.

enum nvinfer1::TensorLocation : int

strong

The location for tensor data storage, device or host.

Enumerator
kDEVICE	Data stored on device.
kHOST	Data stored on host.

enum nvinfer1::TopKOperation : int

strong

Enumerates the operations that may be performed by a TopK layer.

Enumerator
kMAX	Maximum of the elements.
kMIN	Minimum of the elements.

enum nvinfer1::UnaryOperation : int

strong

Enumerates the unary operations that may be performed by a Unary layer.

See Also

IUnaryLayer

Enumerator
kEXP	Exponentiation.
kLOG	Log (base e).
kSQRT	Square root.
kRECIP	Reciprocal.
kABS	Absolute value.
kNEG	Negation.
kSIN	Sine.
kCOS	Cosine.
kTAN	Tangent.
kSINH	Hyperbolic sine.
kCOSH	Hyperbolic cosine.
kASIN	Inverse sine.
kACOS	Inverse cosine.
kATAN	Inverse tangent.
kASINH	Inverse hyperbolic sine.
kACOSH	Inverse hyperbolic cosine.
kATANH	Inverse hyperbolic tangent.
kCEIL	Ceiling.
kFLOOR	Floor.

enum nvinfer1::WeightsRole : int

strong

How a layer uses particular Weights.

The power weights of an IScaleLayer are omitted. Refitting those is not supported.

Enumerator
kKERNEL	kernel for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer
kBIAS	bias for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer
kSHIFT	shift part of IScaleLayer
kSCALE	scale part of IScaleLayer
kCONSTANT	weights for IConstantLayer

Function Documentation

template<>

int nvinfer1::EnumMax< ActivationType > ( )

inline

Maximum number of elements in ActivationType enum.

See Also

ActivationType

template<>

int nvinfer1::EnumMax< CalibrationAlgoType > ( )

inline

Maximum number of elements in CalibrationAlgoType enum.

See Also

DataType

template<>

int nvinfer1::EnumMax< DataType > ( )

inline

Maximum number of elements in DataType enum.

See Also

DataType

template<>

int nvinfer1::EnumMax< DeviceType > ( )

inline

Maximum number of elements in DeviceType enum.

See Also

DeviceType

template<>

int nvinfer1::EnumMax< DimensionType > ( )

inline

Maximum number of elements in DimensionType enum.

See Also

DimensionType

template<>

int nvinfer1::EnumMax< ElementWiseOperation > ( )

inline

Maximum number of elements in ElementWiseOperation enum.

See Also

ElementWiseOperation

template<>

int nvinfer1::EnumMax< EngineCapability > ( )

inline

Maximum number of elements in EngineCapability enum.

See Also

EngineCapability

template<>

int nvinfer1::EnumMax< ILogger::Severity > ( )

inline

Maximum number of elements in ILogger::Severity enum.

See Also

ILogger::Severity

template<>

int nvinfer1::EnumMax< LayerType > ( )

inline

Maximum number of elements in LayerType enum.

See Also

LayerType

template<>

int nvinfer1::EnumMax< MatrixOperation > ( )

inline

Maximum number of elements in MatrixOperation enum.

See Also

DataType

template<>

int nvinfer1::EnumMax< PaddingMode > ( )

inline

Maximum number of elements in PaddingMode enum.

See Also

PaddingMode

template<>

int nvinfer1::EnumMax< PluginFormat > ( )

inline

Maximum number of elements in PluginFormat enum.

See Also

PluginFormat

template<>

int nvinfer1::EnumMax< PoolingType > ( )

inline

Maximum number of elements in PoolingType enum.

See Also

PoolingType

template<>

int nvinfer1::EnumMax< ReduceOperation > ( )

inline

Maximum number of elements in ReduceOperation enum.

See Also

ReduceOperation

template<>

int nvinfer1::EnumMax< RNNDirection > ( )

inline

Maximum number of elements in RNNDirection enum.

See Also

RNNDirection

template<>

int nvinfer1::EnumMax< RNNGateType > ( )

inline

Maximum number of elements in RNNGateType enum.

See Also

RNNGateType

template<>

int nvinfer1::EnumMax< RNNInputMode > ( )

inline

Maximum number of elements in RNNInputMode enum.

See Also

RNNInputMode

template<>

int nvinfer1::EnumMax< RNNOperation > ( )

inline

Maximum number of elements in RNNOperation enum.

See Also

RNNOperation

template<>

int nvinfer1::EnumMax< ScaleMode > ( )

inline

Maximum number of elements in ScaleMode enum.

See Also

ScaleMode

template<>

int nvinfer1::EnumMax< TensorLocation > ( )

inline

Maximum number of elements in TensorLocation enum.

See Also

TensorLocation

template<>

int nvinfer1::EnumMax< TopKOperation > ( )

inline

Maximum number of elements in TopKOperation enum.

See Also

TopKOperation

template<>

int nvinfer1::EnumMax< UnaryOperation > ( )

inline

Maximum number of elements in UnaryOperation enum.

See Also

UnaryOperation

template<>

int nvinfer1::EnumMax< WeightsRole > ( )

inline

Maximum number of elements in WeightsRole enum.

See Also

WeightsRole