TensorRT 8.2.1
nvinfer1::IRNNv2Layer Class Reference

An RNN layer in a network definition, version 2. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IRNNv2Layer:
nvinfer1::ILayer nvinfer1::INoCopy

Public Member Functions

int32_t getLayerCount () const noexcept
 Get the layer count of the RNN.
 
int32_t getHiddenSize () const noexcept
 Get the hidden size of the RNN.
 
int32_t getMaxSeqLength () const noexcept
 Get the maximum sequence length of the RNN.
 
int32_t getDataLength () const noexcept
 Get the maximum data length of the RNN.
 
void setSequenceLengths (ITensor &seqLengths) noexcept
 Specify individual sequence lengths in the batch with the ITensor pointed to by seqLengths. More...
 
ITensorgetSequenceLengths () const noexcept
 Get the sequence lengths specified for the RNN. More...
 
void setOperation (RNNOperation op) noexcept
 Set the operation of the RNN layer. More...
 
RNNOperation getOperation () const noexcept
 Get the operation of the RNN layer. More...
 
void setInputMode (RNNInputMode op) noexcept
 Set the input mode of the RNN layer. More...
 
RNNInputMode getInputMode () const noexcept
 Get the input mode of the RNN layer. More...
 
void setDirection (RNNDirection op) noexcept
 Set the direction of the RNN layer. More...
 
RNNDirection getDirection () const noexcept
 Get the direction of the RNN layer. More...
 
void setWeightsForGate (int32_t layerIndex, RNNGateType gate, bool isW, Weights weights) noexcept
 Set the weight parameters for an individual gate in the RNN. More...
 
Weights getWeightsForGate (int32_t layerIndex, RNNGateType gate, bool isW) const noexcept
 Get the weight parameters for an individual gate in the RNN. More...
 
void setBiasForGate (int32_t layerIndex, RNNGateType gate, bool isW, Weights bias) noexcept
 Set the bias parameters for an individual gate in the RNN. More...
 
Weights getBiasForGate (int32_t layerIndex, RNNGateType gate, bool isW) const noexcept
 Get the bias parameters for an individual gate in the RNN. More...
 
void setHiddenState (ITensor &hidden) noexcept
 Set the initial hidden state of the RNN with the provided hidden ITensor. More...
 
ITensorgetHiddenState () const noexcept
 Get the initial hidden state of the RNN. More...
 
void setCellState (ITensor &cell) noexcept
 Set the initial cell state of the LSTM with the provided cell ITensor. More...
 
ITensorgetCellState () const noexcept
 Get the initial cell state of the RNN. More...
 
- Public Member Functions inherited from nvinfer1::ILayer
LayerType getType () const noexcept
 Return the type of a layer. More...
 
void setName (const char *name) noexcept
 Set the name of a layer. More...
 
const char * getName () const noexcept
 Return the name of a layer. More...
 
int32_t getNbInputs () const noexcept
 Get the number of inputs of a layer.
 
ITensorgetInput (int32_t index) const noexcept
 Get the layer input corresponding to the given index. More...
 
int32_t getNbOutputs () const noexcept
 Get the number of outputs of a layer.
 
ITensorgetOutput (int32_t index) const noexcept
 Get the layer output corresponding to the given index. More...
 
void setInput (int32_t index, ITensor &tensor) noexcept
 Replace an input of this layer with a specific tensor. More...
 
void setPrecision (DataType dataType) noexcept
 Set the computational precision of this layer. More...
 
DataType getPrecision () const noexcept
 get the computational precision of this layer More...
 
bool precisionIsSet () const noexcept
 whether the computational precision has been set for this layer More...
 
void resetPrecision () noexcept
 reset the computational precision for this layer More...
 
void setOutputType (int32_t index, DataType dataType) noexcept
 Set the output type of this layer. More...
 
DataType getOutputType (int32_t index) const noexcept
 get the output type of this layer More...
 
bool outputTypeIsSet (int32_t index) const noexcept
 whether the output type has been set for this layer More...
 
void resetOutputType (int32_t index) noexcept
 reset the output type for this layer More...
 

Protected Attributes

apiv::VRNNv2Layer * mImpl
 
- Protected Attributes inherited from nvinfer1::ILayer
apiv::VLayer * mLayer
 

Additional Inherited Members

- Protected Member Functions inherited from nvinfer1::INoCopy
 INoCopy (const INoCopy &other)=delete
 
INoCopyoperator= (const INoCopy &other)=delete
 
 INoCopy (INoCopy &&other)=delete
 
INoCopyoperator= (INoCopy &&other)=delete
 

Detailed Description

An RNN layer in a network definition, version 2.

This layer supersedes IRNNLayer.

Deprecated:
IRNNv2Layer will be removed in TensorRT 9.0, use INetworkDefinition::addLoop instead.
Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

◆ getBiasForGate()

Weights nvinfer1::IRNNv2Layer::getBiasForGate ( int32_t  layerIndex,
RNNGateType  gate,
bool  isW 
) const
inlinenoexcept

Get the bias parameters for an individual gate in the RNN.

See also
setBiasForGate()

◆ getCellState()

ITensor * nvinfer1::IRNNv2Layer::getCellState ( ) const
inlinenoexcept

Get the initial cell state of the RNN.

See also
setCellState()

◆ getDirection()

RNNDirection nvinfer1::IRNNv2Layer::getDirection ( ) const
inlinenoexcept

Get the direction of the RNN layer.

See also
setDirection(), RNNDirection

◆ getHiddenState()

ITensor * nvinfer1::IRNNv2Layer::getHiddenState ( ) const
inlinenoexcept

Get the initial hidden state of the RNN.

See also
setHiddenState()

◆ getInputMode()

RNNInputMode nvinfer1::IRNNv2Layer::getInputMode ( ) const
inlinenoexcept

Get the input mode of the RNN layer.

See also
setInputMode(), RNNInputMode

◆ getOperation()

RNNOperation nvinfer1::IRNNv2Layer::getOperation ( ) const
inlinenoexcept

Get the operation of the RNN layer.

See also
setOperation(), RNNOperation

◆ getSequenceLengths()

ITensor * nvinfer1::IRNNv2Layer::getSequenceLengths ( ) const
inlinenoexcept

Get the sequence lengths specified for the RNN.

Returns
nullptr if no sequence lengths were specified, the sequence length data otherwise.
See also
setSequenceLengths()

◆ getWeightsForGate()

Weights nvinfer1::IRNNv2Layer::getWeightsForGate ( int32_t  layerIndex,
RNNGateType  gate,
bool  isW 
) const
inlinenoexcept

Get the weight parameters for an individual gate in the RNN.

See also
setWeightsForGate()

◆ setBiasForGate()

void nvinfer1::IRNNv2Layer::setBiasForGate ( int32_t  layerIndex,
RNNGateType  gate,
bool  isW,
Weights  bias 
)
inlinenoexcept

Set the bias parameters for an individual gate in the RNN.

The DataType for this structure must be ::kFLOAT or ::kHALF, and must be the same datatype as the input tensor.

Each bias vector has a fixed size, getHiddenSize().

Parameters
layerIndexThe index of the layer that contains this gate. See the section Order of weight matrices in IRNNLayer::setWeights() for a description of the layer index.
gateThe name of the gate within the RNN layer. The gate name must correspond to one of the gates used by this layer's RNNOperation.
isWTrue if the bias parameters are for the input bias Wb[g] and false if they are for the recurrent input bias Rb[g]. See RNNOperation for equations showing how these bias vectors are used in the RNN gate.
biasThe weight structure holding the bias parameters, which should be an array of size getHiddenSize().

◆ setCellState()

void nvinfer1::IRNNv2Layer::setCellState ( ITensor cell)
inlinenoexcept

Set the initial cell state of the LSTM with the provided cell ITensor.

The cell ITensor should have the dimensions {N1, ..., Np, L, H}, where:

  • N1..Np are the index dimensions specified by the input tensor
  • L is the number of layers in the RNN, equal to getLayerCount() if getDirection is ::kUNIDIRECTION, and 2x getLayerCount() if getDirection is ::kBIDIRECTION. In the bi-directional case, layer l's final forward hidden state is stored in L = 2*l, and final backward hidden state is stored in L= 2*l + 1.
  • H is the hidden state for each layer, equal to getHiddenSize().

It is an error to call setCellState() on an RNN layer that is not configured with RNNOperation::kLSTM.

◆ setDirection()

void nvinfer1::IRNNv2Layer::setDirection ( RNNDirection  op)
inlinenoexcept

Set the direction of the RNN layer.

The direction determines if the RNN is run as a unidirectional(left to right) or bidirectional(left to right and right to left). In the ::kBIDIRECTION case the output is concatenated together, resulting in output size of 2x getHiddenSize().

See also
getDirection(), RNNDirection

◆ setHiddenState()

void nvinfer1::IRNNv2Layer::setHiddenState ( ITensor hidden)
inlinenoexcept

Set the initial hidden state of the RNN with the provided hidden ITensor.

The hidden ITensor should have the dimensions {N1, ..., Np, L, H}, where:

  • N1..Np are the index dimensions specified by the input tensor
  • L is the number of layers in the RNN, equal to getLayerCount() if getDirection is ::kUNIDIRECTION, and 2x getLayerCount() if getDirection is ::kBIDIRECTION. In the bi-directional case, layer l's final forward hidden state is stored in L = 2*l, and final backward hidden state is stored in L= 2*l + 1.
  • H is the hidden state for each layer, equal to getHiddenSize().

◆ setInputMode()

void nvinfer1::IRNNv2Layer::setInputMode ( RNNInputMode  op)
inlinenoexcept

Set the input mode of the RNN layer.

See also
getInputMode(), RNNInputMode

◆ setOperation()

void nvinfer1::IRNNv2Layer::setOperation ( RNNOperation  op)
inlinenoexcept

Set the operation of the RNN layer.

See also
getOperation(), RNNOperation

◆ setSequenceLengths()

void nvinfer1::IRNNv2Layer::setSequenceLengths ( ITensor seqLengths)
inlinenoexcept

Specify individual sequence lengths in the batch with the ITensor pointed to by seqLengths.

The seqLengths ITensor should be a {N1, ..., Np} tensor, where N1..Np are the index dimensions of the input tensor to the RNN.

If this is not specified, then the RNN layer assumes all sequences are size getMaxSeqLength().

All sequence lengths in seqLengths should be in the range [1, getMaxSeqLength()]. Zero-length sequences are not supported.

This tensor must be of type DataType::kINT32.

◆ setWeightsForGate()

void nvinfer1::IRNNv2Layer::setWeightsForGate ( int32_t  layerIndex,
RNNGateType  gate,
bool  isW,
Weights  weights 
)
inlinenoexcept

Set the weight parameters for an individual gate in the RNN.

The DataType for this structure must be ::kFLOAT or ::kHALF, and must be the same datatype as the input tensor.

Each parameter matrix is row-major in memory, and has the following dimensions:

Let K := { ::kUNIDIRECTION => 1
{ ::kBIDIRECTION => 2
l := layer index (as described above)
E := getDataLength() (the embedding length)
isW := true if the matrix is an input (W) matrix, and false if
the matrix is a recurrent input (R) matrix.
if isW:
if l < K and ::kSKIP:
(numRows, numCols) := (0, 0) # input matrix is skipped
elif l < K and ::kLINEAR:
(numRows, numCols) := (H, E) # input matrix acts on input data size E
elif l >= K:
(numRows, numCols) := (H, K * H) # input matrix acts on previous hidden state
else: # not isW
(numRows, numCols) := (H, H)
int32_t getDataLength() const noexcept
Get the maximum data length of the RNN.
Definition: NvInfer.h:3267
int32_t getHiddenSize() const noexcept
Get the hidden size of the RNN.
Definition: NvInfer.h:3259

In other words, the input weights of the first layer of the RNN (if not skipped) transform a getDataLength()-size column vector into a getHiddenSize()-size column vector. The input weights of subsequent layers transform a K*getHiddenSize()-size column vector into a getHiddenSize()-size column vector. K=2 in the bidirectional case to account for the full hidden state being the concatenation of the forward and backward RNN hidden states.

The recurrent weight matrices for all layers all have shape (H, H), both in the unidirectional and bidirectional cases. (In the bidirectional case, each recurrent weight matrix for the (forward or backward) RNN cell operates on the previous (forward or backward) RNN cell's hidden state, which is size H).

Parameters
layerIndexThe index of the layer that contains this gate. See the section
gateThe name of the gate within the RNN layer. The gate name must correspond to one of the gates used by this layer's RNNOperation.
isWTrue if the weight parameters are for the input matrix W[g] and false if they are for the recurrent input matrix R[g]. See RNNOperation for equations showing how these matrices are used in the RNN gate.
weightsThe weight structure holding the weight parameters, which are stored as a row-major 2D matrix. See the layout of elements within a weight matrix in IRNNLayer::setWeights() for documentation on the expected dimensions of this matrix.

The documentation for this class was generated from the following file: