An RNN layer in a network definition, version 2. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IRNNv2Layer:

Public Member Functions
int32_t	getLayerCount () const noexcept
	Get the layer count of the RNN. More...

int32_t	getHiddenSize () const noexcept
	Get the hidden size of the RNN. More...

int32_t	getMaxSeqLength () const noexcept
	Get the maximum sequence length of the RNN. More...

int32_t	getDataLength () const noexcept
	Get the maximum data length of the RNN. More...

void	setSequenceLengths (ITensor &seqLengths) noexcept
	Specify individual sequence lengths in the batch with the ITensor pointed to by `seqLengths`. More...

ITensor *	getSequenceLengths () const noexcept
	Get the sequence lengths specified for the RNN. More...

void	setOperation (RNNOperation op) noexcept
	Set the operation of the RNN layer. More...

RNNOperation	getOperation () const noexcept
	Get the operation of the RNN layer. More...

void	setInputMode (RNNInputMode op) noexcept
	Set the input mode of the RNN layer. More...

RNNInputMode	getInputMode () const noexcept
	Get the input mode of the RNN layer. More...

void	setDirection (RNNDirection op) noexcept
	Set the direction of the RNN layer. More...

RNNDirection	getDirection () const noexcept
	Get the direction of the RNN layer. More...

void	setWeightsForGate (int32_t layerIndex, RNNGateType gate, bool isW, Weights weights) noexcept
	Set the weight parameters for an individual gate in the RNN. More...

Weights	getWeightsForGate (int32_t layerIndex, RNNGateType gate, bool isW) const noexcept
	Get the weight parameters for an individual gate in the RNN. More...

void	setBiasForGate (int32_t layerIndex, RNNGateType gate, bool isW, Weights bias) noexcept
	Set the bias parameters for an individual gate in the RNN. More...

Weights	getBiasForGate (int32_t layerIndex, RNNGateType gate, bool isW) const noexcept
	Get the bias parameters for an individual gate in the RNN. More...

void	setHiddenState (ITensor &hidden) noexcept
	Set the initial hidden state of the RNN with the provided `hidden` ITensor. More...

ITensor *	getHiddenState () const noexcept
	Get the initial hidden state of the RNN. More...

void	setCellState (ITensor &cell) noexcept
	Set the initial cell state of the LSTM with the provided `cell` ITensor. More...

ITensor *	getCellState () const noexcept
	Get the initial cell state of the RNN. More...

Public Member Functions inherited from nvinfer1::ILayer
LayerType	getType () const noexcept
	Return the type of a layer. More...

void	setName (char const *name) noexcept
	Set the name of a layer. More...

char const *	getName () const noexcept
	Return the name of a layer. More...

int32_t	getNbInputs () const noexcept
	Get the number of inputs of a layer. More...

ITensor *	getInput (int32_t index) const noexcept
	Get the layer input corresponding to the given index. More...

int32_t	getNbOutputs () const noexcept
	Get the number of outputs of a layer. More...

ITensor *	getOutput (int32_t index) const noexcept
	Get the layer output corresponding to the given index. More...

void	setInput (int32_t index, ITensor &tensor) noexcept
	Replace an input of this layer with a specific tensor. More...

void	setPrecision (DataType dataType) noexcept
	Set the computational precision of this layer. More...

DataType	getPrecision () const noexcept
	get the computational precision of this layer More...

bool	precisionIsSet () const noexcept
	whether the computational precision has been set for this layer More...

void	resetPrecision () noexcept
	reset the computational precision for this layer More...

void	setOutputType (int32_t index, DataType dataType) noexcept
	Set the output type of this layer. More...

DataType	getOutputType (int32_t index) const noexcept
	get the output type of this layer More...

bool	outputTypeIsSet (int32_t index) const noexcept
	whether the output type has been set for this layer More...

void	resetOutputType (int32_t index) noexcept
	reset the output type for this layer More...

Protected Member Functions
virtual	~IRNNv2Layer () noexcept=default

Protected Member Functions inherited from nvinfer1::ILayer
virtual	~ILayer () noexcept=default

Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default

virtual	~INoCopy ()=default

	INoCopy (INoCopy const &other)=delete

INoCopy &	operator= (INoCopy const &other)=delete

	INoCopy (INoCopy &&other)=delete

INoCopy &	operator= (INoCopy &&other)=delete

Protected Attributes
apiv::VRNNv2Layer *	mImpl

Protected Attributes inherited from nvinfer1::ILayer
apiv::VLayer *	mLayer

Detailed Description

An RNN layer in a network definition, version 2.

This layer supersedes IRNNLayer.

Deprecated:: Deprecated prior to TensorRT 8.0 and will be removed in 9.0. Superseded by INetworkDefinition::addLoop().

Warning: Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRNNv2Layer()

virtual nvinfer1::IRNNv2Layer::~IRNNv2Layer ( )

protectedvirtualdefaultnoexcept

Member Function Documentation

◆ getBiasForGate()

Weights nvinfer1::IRNNv2Layer::getBiasForGate	(	int32_t	layerIndex,
		RNNGateType	gate,
		bool	isW
	)		const

inlinenoexcept

Get the bias parameters for an individual gate in the RNN.

See also: setBiasForGate()

◆ getCellState()

ITensor * nvinfer1::IRNNv2Layer::getCellState ( ) const

inlinenoexcept

Get the initial cell state of the RNN.

See also: setCellState()

◆ getDataLength()

int32_t nvinfer1::IRNNv2Layer::getDataLength ( ) const

inlinenoexcept

Get the maximum data length of the RNN.

◆ getDirection()

RNNDirection nvinfer1::IRNNv2Layer::getDirection ( ) const

inlinenoexcept

Get the direction of the RNN layer.

See also: setDirection(), RNNDirection

◆ getHiddenSize()

int32_t nvinfer1::IRNNv2Layer::getHiddenSize ( ) const

inlinenoexcept

Get the hidden size of the RNN.

◆ getHiddenState()

ITensor * nvinfer1::IRNNv2Layer::getHiddenState ( ) const

inlinenoexcept

Get the initial hidden state of the RNN.

See also: setHiddenState()

◆ getInputMode()

RNNInputMode nvinfer1::IRNNv2Layer::getInputMode ( ) const

inlinenoexcept

Get the input mode of the RNN layer.

See also: setInputMode(), RNNInputMode

◆ getLayerCount()

int32_t nvinfer1::IRNNv2Layer::getLayerCount ( ) const

inlinenoexcept

Get the layer count of the RNN.

◆ getMaxSeqLength()

int32_t nvinfer1::IRNNv2Layer::getMaxSeqLength ( ) const

inlinenoexcept

Get the maximum sequence length of the RNN.

◆ getOperation()

RNNOperation nvinfer1::IRNNv2Layer::getOperation ( ) const

inlinenoexcept

Get the operation of the RNN layer.

See also: setOperation(), RNNOperation

◆ getSequenceLengths()

ITensor * nvinfer1::IRNNv2Layer::getSequenceLengths ( ) const

inlinenoexcept

Get the sequence lengths specified for the RNN.

Returns: nullptr if no sequence lengths were specified, the sequence length data otherwise.

See also: setSequenceLengths()

◆ getWeightsForGate()

Weights nvinfer1::IRNNv2Layer::getWeightsForGate	(	int32_t	layerIndex,
		RNNGateType	gate,
		bool	isW
	)		const

inlinenoexcept

Get the weight parameters for an individual gate in the RNN.

See also: setWeightsForGate()

◆ setBiasForGate()

void nvinfer1::IRNNv2Layer::setBiasForGate	(	int32_t	layerIndex,
		RNNGateType	gate,
		bool	isW,
		Weights	bias
	)

inlinenoexcept

Set the bias parameters for an individual gate in the RNN.

The DataType for this structure must be ::kFLOAT or ::kHALF, and must be the same datatype as the input tensor.

Each bias vector has a fixed size, getHiddenSize().

Parameters

layerIndex	The index of the layer that contains this gate. See the section Order of weight matrices in IRNNLayer::setWeights() for a description of the layer index.
gate	The name of the gate within the RNN layer. The gate name must correspond to one of the gates used by this layer's RNNOperation.
isW	True if the bias parameters are for the input bias Wb[g] and false if they are for the recurrent input bias Rb[g]. See RNNOperation for equations showing how these bias vectors are used in the RNN gate.
bias	The weight structure holding the bias parameters, which should be an array of size getHiddenSize().

◆ setCellState()

void nvinfer1::IRNNv2Layer::setCellState ( ITensor & cell )

inlinenoexcept

Set the initial cell state of the LSTM with the provided cell ITensor.

The cell ITensor should have the dimensions {N1, ..., Np, L, H}, where:

N1..Np are the index dimensions specified by the input tensor
L is the number of layers in the RNN, equal to getLayerCount() if getDirection is ::kUNIDIRECTION, and 2x getLayerCount() if getDirection is ::kBIDIRECTION. In the bi-directional case, layer l's final forward hidden state is stored in L = 2*l, and final backward hidden state is stored in L= 2*l + 1.
H is the hidden state for each layer, equal to getHiddenSize().

It is an error to call setCellState() on an RNN layer that is not configured with RNNOperation::kLSTM.

◆ setDirection()

void nvinfer1::IRNNv2Layer::setDirection ( RNNDirection op )

inlinenoexcept

Set the direction of the RNN layer.

The direction determines if the RNN is run as a unidirectional(left to right) or bidirectional(left to right and right to left). In the ::kBIDIRECTION case the output is concatenated together, resulting in output size of 2x getHiddenSize().

See also: getDirection(), RNNDirection

◆ setHiddenState()

void nvinfer1::IRNNv2Layer::setHiddenState ( ITensor & hidden )

inlinenoexcept

Set the initial hidden state of the RNN with the provided hidden ITensor.

The hidden ITensor should have the dimensions {N1, ..., Np, L, H}, where:

N1..Np are the index dimensions specified by the input tensor
L is the number of layers in the RNN, equal to getLayerCount() if getDirection is ::kUNIDIRECTION, and 2x getLayerCount() if getDirection is ::kBIDIRECTION. In the bi-directional case, layer l's final forward hidden state is stored in L = 2*l, and final backward hidden state is stored in L= 2*l + 1.
H is the hidden state for each layer, equal to getHiddenSize().

◆ setInputMode()

void nvinfer1::IRNNv2Layer::setInputMode ( RNNInputMode op )

inlinenoexcept

Set the input mode of the RNN layer.

See also: getInputMode(), RNNInputMode

◆ setOperation()

void nvinfer1::IRNNv2Layer::setOperation ( RNNOperation op )

inlinenoexcept

Set the operation of the RNN layer.

See also: getOperation(), RNNOperation

◆ setSequenceLengths()

void nvinfer1::IRNNv2Layer::setSequenceLengths ( ITensor & seqLengths )

inlinenoexcept

Specify individual sequence lengths in the batch with the ITensor pointed to by seqLengths.

The seqLengths ITensor should be a {N1, ..., Np} tensor, where N1..Np are the index dimensions of the input tensor to the RNN.

If this is not specified, then the RNN layer assumes all sequences are size getMaxSeqLength().

All sequence lengths in seqLengths should be in the range [1, getMaxSeqLength()]. Zero-length sequences are not supported.

This tensor must be of type DataType::kINT32.

◆ setWeightsForGate()

void nvinfer1::IRNNv2Layer::setWeightsForGate	(	int32_t	layerIndex,
		RNNGateType	gate,
		bool	isW,
		Weights	weights
	)

inlinenoexcept

Set the weight parameters for an individual gate in the RNN.

The DataType for this structure must be ::kFLOAT or ::kHALF, and must be the same datatype as the input tensor.

Each parameter matrix is row-major in memory, and has the following dimensions:

 Let K := { ::kUNIDIRECTION => 1
          { ::kBIDIRECTION => 2
     l := layer index (as described above)
     H := getHiddenSize()
     E := getDataLength() (the embedding length)
     isW := true if the matrix is an input (W) matrix, and false if
            the matrix is a recurrent input (R) matrix.
 
if isW:
   if l < K and ::kSKIP:
      (numRows, numCols) := (0, 0) # input matrix is skipped
   elif l < K and ::kLINEAR:
      (numRows, numCols) := (H, E) # input matrix acts on input data size E
   elif l >= K:
      (numRows, numCols) := (H, K * H) # input matrix acts on previous hidden state
else: # not isW
   (numRows, numCols) := (H, H)

In other words, the input weights of the first layer of the RNN (if not skipped) transform a getDataLength()-size column vector into a getHiddenSize()-size column vector. The input weights of subsequent layers transform a K*getHiddenSize()-size column vector into a getHiddenSize()-size column vector. K=2 in the bidirectional case to account for the full hidden state being the concatenation of the forward and backward RNN hidden states.

The recurrent weight matrices for all layers all have shape (H, H), both in the unidirectional and bidirectional cases. (In the bidirectional case, each recurrent weight matrix for the (forward or backward) RNN cell operates on the previous (forward or backward) RNN cell's hidden state, which is size H).

Parameters

layerIndex	The index of the layer that contains this gate. See the section
gate	The name of the gate within the RNN layer. The gate name must correspond to one of the gates used by this layer's RNNOperation.
isW	True if the weight parameters are for the input matrix W[g] and false if they are for the recurrent input matrix R[g]. See RNNOperation for equations showing how these matrices are used in the RNN gate.
weights	The weight structure holding the weight parameters, which are stored as a row-major 2D matrix. See the layout of elements within a weight matrix in IRNNLayer::setWeights() for documentation on the expected dimensions of this matrix.

Member Data Documentation

◆ mImpl

apiv::VRNNv2Layer* nvinfer1::IRNNv2Layer::mImpl

protected

The documentation for this class was generated from the following file:

NvInfer.h

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ ~IRNNv2Layer()

Member Function Documentation

◆ getBiasForGate()

◆ getCellState()

◆ getDataLength()

◆ getDirection()

◆ getHiddenSize()

◆ getHiddenState()

◆ getInputMode()

◆ getLayerCount()

◆ getMaxSeqLength()

◆ getOperation()

◆ getSequenceLengths()

◆ getWeightsForGate()

◆ setBiasForGate()

◆ setCellState()

◆ setDirection()

◆ setHiddenState()

◆ setInputMode()

◆ setOperation()

◆ setSequenceLengths()

◆ setWeightsForGate()

Member Data Documentation

◆ mImpl