TensorRT  7.0.0.11
nvinfer1::IRNNLayer Class Referenceabstract

A RNN layer in a network definition. More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IRNNLayer:
nvinfer1::ILayer

Public Member Functions

virtual unsigned getLayerCount () const =0
 Get the number of layers in the RNN. More...
 
virtual std::size_t getHiddenSize () const =0
 Get the size of the hidden layers. More...
 
virtual int getSeqLength () const =0
 Get the sequence length. More...
 
virtual void setOperation (RNNOperation op)=0
 Set the operation of the RNN layer. More...
 
virtual RNNOperation getOperation () const =0
 Get the operation of the RNN layer. More...
 
virtual void setInputMode (RNNInputMode op)=0
 Set the operation of the RNN layer. More...
 
virtual RNNInputMode getInputMode () const =0
 Get the operation of the RNN layer. More...
 
virtual void setDirection (RNNDirection op)=0
 Set the direction of the RNN layer. More...
 
virtual RNNDirection getDirection () const =0
 Get the direction of the RNN layer. More...
 
virtual void setWeights (Weights weights)=0
 Set the weight parameters for the RNN. More...
 
virtual Weights getWeights () const =0
 Get the W weights for the RNN. More...
 
virtual void setBias (Weights bias)=0
 Set the bias parameters for the RNN. More...
 
virtual Weights getBias () const =0
 Get the bias parameter vector for the RNN. More...
 
virtual int getDataLength () const =0
 Get the length of the data being processed by the RNN for use in computing other values. More...
 
virtual void setHiddenState (ITensor &hidden)=0
 Set the initial hidden state of the RNN with the provided hidden ITensor. More...
 
virtual ITensorgetHiddenState () const =0
 Get the initial hidden state of the RNN. More...
 
virtual void setCellState (ITensor &cell)=0
 Set the initial cell state of the RNN with the provided cell ITensor. More...
 
virtual ITensorgetCellState () const =0
 Get the initial cell state of the RNN. More...
 
- Public Member Functions inherited from nvinfer1::ILayer
virtual LayerType getType () const =0
 Return the type of a layer. More...
 
virtual void setName (const char *name)=0
 Set the name of a layer. More...
 
virtual const char * getName () const =0
 Return the name of a layer. More...
 
virtual int getNbInputs () const =0
 Get the number of inputs of a layer.
 
virtual ITensorgetInput (int index) const =0
 Get the layer input corresponding to the given index. More...
 
virtual int getNbOutputs () const =0
 Get the number of outputs of a layer.
 
virtual ITensorgetOutput (int index) const =0
 Get the layer output corresponding to the given index. More...
 
virtual void setInput (int index, ITensor &tensor)=0
 Replace an input of this layer with a specific tensor. More...
 
virtual void setPrecision (DataType dataType)=0
 Set the computational precision of this layer. More...
 
virtual DataType getPrecision () const =0
 get the computational precision of this layer More...
 
virtual bool precisionIsSet () const =0
 whether the computational precision has been set for this layer More...
 
virtual void resetPrecision ()=0
 reset the computational precision for this layer More...
 
virtual void setOutputType (int index, DataType dataType)=0
 Set the output type of this layer. More...
 
virtual DataType getOutputType (int index) const =0
 get the output type of this layer More...
 
virtual bool outputTypeIsSet (int index) const =0
 whether the output type has been set for this layer More...
 
virtual void resetOutputType (int index)=0
 reset the output type for this layer More...
 

Detailed Description

A RNN layer in a network definition.

This layer applies an RNN operation on the inputs. This layer only works with networks that that have an implicit batch dimension. For dynamic shapes and explicit batch dimension networks, use IRNNv2Layer.

Deprecated:
This interface is superseded by IRNNv2Layer.
Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Member Function Documentation

virtual Weights nvinfer1::IRNNLayer::getBias ( ) const
pure virtual

Get the bias parameter vector for the RNN.

See also
setBias()
virtual ITensor* nvinfer1::IRNNLayer::getCellState ( ) const
pure virtual

Get the initial cell state of the RNN.

Returns
nullptr if no initial cell tensor was specified, the initial cell data otherwise.
virtual int nvinfer1::IRNNLayer::getDataLength ( ) const
pure virtual

Get the length of the data being processed by the RNN for use in computing other values.

See also
setHiddenState(), setCellState()
virtual RNNDirection nvinfer1::IRNNLayer::getDirection ( ) const
pure virtual

Get the direction of the RNN layer.

See also
setDirection(), RNNDirection
virtual std::size_t nvinfer1::IRNNLayer::getHiddenSize ( ) const
pure virtual

Get the size of the hidden layers.

The hidden size is the value of hiddenSize parameter passed into addRNN().

Returns
The internal hidden layer size for the RNN.
See also
getDirection(), addRNN()
virtual ITensor* nvinfer1::IRNNLayer::getHiddenState ( ) const
pure virtual

Get the initial hidden state of the RNN.

Returns
nullptr if no initial hidden tensor was specified, the initial hidden data otherwise.
virtual RNNInputMode nvinfer1::IRNNLayer::getInputMode ( ) const
pure virtual

Get the operation of the RNN layer.

See also
setInputMode(), RNNInputMode
virtual unsigned nvinfer1::IRNNLayer::getLayerCount ( ) const
pure virtual

Get the number of layers in the RNN.

Returns
The number of layers in the RNN.
virtual RNNOperation nvinfer1::IRNNLayer::getOperation ( ) const
pure virtual

Get the operation of the RNN layer.

See also
setOperation(), RNNOperation
virtual int nvinfer1::IRNNLayer::getSeqLength ( ) const
pure virtual

Get the sequence length.

The sequence length is the maximum number of time steps passed into the addRNN() function. This is also the maximum number of input tensors that the RNN can process at once.

Returns
the maximum number of time steps that can be executed by a single call RNN layer.
virtual Weights nvinfer1::IRNNLayer::getWeights ( ) const
pure virtual

Get the W weights for the RNN.

See also
setWeights()
virtual void nvinfer1::IRNNLayer::setBias ( Weights  bias)
pure virtual

Set the bias parameters for the RNN.

Parameters
biasThe weight structure holding the bias parameters.

The trained weights for the bias parameter vectors of the RNN. The DataType for this structure must be kFLOAT or kHALF, and must be the same datatype as the input tensor.

The layout of the weight structure depends on the RNNOperation, RNNInputMode, and RNNDirection of the layer. The array specified by weights.values contains a sequence of bias vectors, where each bias vector is linearly appended after the previous without padding; e.g., if bias vector 0 and 1 have M and N elements respectively, then the layout of weights.values in memory looks like:

index | 0 1 2 3 4 ... M-2 M-1 | M M+1 ... M+N-2 M+N-1 | M+N M+N+1 M+N+2 ... | ...
data |-- bias vector 0 --|-- bias vector 1 --|-- bias vector 2 --| ...

The ordering of bias vectors is similar to the ordering of weight matrices as described in setWeights(). To determine the order of bias vectors for a given RNN configuration, determine the ordered list of weight matrices [ W0, W1, ..., Wn ]. Then replace each weight matrix with its corresponding bias vector, i.e. apply the following transform (for layer l, gate g):

  • Wl[g] becomes Wbl[g]
  • Rl[g] becomes Rbl[g]

For example:

Each bias vector has a fixed size, getHiddenSize().

See also
getBias(), RNNOperation
virtual void nvinfer1::IRNNLayer::setCellState ( ITensor cell)
pure virtual

Set the initial cell state of the RNN with the provided cell ITensor.

Parameters
cellThe initial cell state of the RNN.

The layout for cell is a linear layout of a 3D matrix:

  • C - The number of layers in the RNN, it must match getLayerCount().
  • H - The number of mini-batches for each time sequence.
  • W - The size of the per layer hidden states, it must match getHiddenSize().

If cell is not specified, then the initial cell state is set to zero.

If getDirection() is kBIDIRECTION, the amount of space required is doubled and C is equal to getLayerCount() * 2.

The cell state only affects LSTM RNN's.

See also
getCellState()
virtual void nvinfer1::IRNNLayer::setDirection ( RNNDirection  op)
pure virtual

Set the direction of the RNN layer.

The direction determines if the RNN is run as a unidirectional(left to right) or bidirectional(left to right and right to left). In the kBIDIRECTION case the output is concatenated together, resulting in output size of 2x getHiddenSize().

See also
getDirection(), RNNDirection
virtual void nvinfer1::IRNNLayer::setHiddenState ( ITensor hidden)
pure virtual

Set the initial hidden state of the RNN with the provided hidden ITensor.

Parameters
hiddenThe initial hidden state of the RNN.

The layout for hidden is a linear layout of a 3D matrix:

  • C - The number of layers in the RNN, it must match getLayerCount().
  • H - The number of mini-batches for each time sequence.
  • W - The size of the per layer hidden states, it must match getHiddenSize().

If getDirection() is kBIDIRECTION, the amount of space required is doubled and C is equal to getLayerCount() * 2.

If hidden is not specified, then the initial hidden state is set to zero.

See also
getHiddenState()
virtual void nvinfer1::IRNNLayer::setInputMode ( RNNInputMode  op)
pure virtual

Set the operation of the RNN layer.

See also
getInputMode(), RNNInputMode
virtual void nvinfer1::IRNNLayer::setOperation ( RNNOperation  op)
pure virtual

Set the operation of the RNN layer.

See also
getOperation(), RNNOperation
virtual void nvinfer1::IRNNLayer::setWeights ( Weights  weights)
pure virtual

Set the weight parameters for the RNN.

Parameters
weightsThe weight structure holding the weight parameters.

The trained weights for the weight parameter matrices of the RNN. The DataType for this structure must be kFLOAT or kHALF, and must be the same datatype as the input tensor.

The layout of the weight structure depends on the RNNOperation, RNNInputMode, and RNNDirection of the layer. The array specified by weights.values contains a sequence of parameter matrices, where each parameter matrix is linearly appended after the previous without padding; e.g., if parameter matrix 0 and 1 have M and N elements respectively, then the layout of weights.values in memory looks like:

index | 0 1 2 3 4 ... M-2 M-1 | M M+1 ... M+N-2 M+N-1 | M+N M+N+1 M+N+2 ... | ...
data |-- parameter matrix 0 --|-- parameter matrix 1 --|-- parameter matrix 2 --| ...

The following sections describe the order of weight matrices and the layout of elements within a weight matrix.

Order of weight matrices

The parameter matrices are ordered as described below:

Let G(op, l) be defined to be a function that produces lists of parameter names, as follows:
G(::kRELU, l) := [ Wl[i], Rl[i] ]
G(::kTANH, l) := [ Wl[i], Rl[i] ]
G(::kLSTM, l) := [ Wl[f], Wl[i], Wl[c], Wl[o], Rl[f], Rl[i], Rl[c], Rl[o] ]
G(::kGRU, l) := [ Wl[z], Wl[r], Wl[h], Rl[z], Rl[r], Rl[h] ]
where Wl[g] and Rl[g] are the names of the input and recurrent
input weight matrices for gate g, layer index l.
See RNNOperation for an overview of the naming convention used for gates.
If getDirection() == ::kUNIDIRECTION, then l identifies the stacked layer of the
RNN, with l=0 being the first recurrent layer and l=L-1 being the last recurrent layer.
If getDirection() == ::kBIDIRECTION, then (l % 2) identifies the direction of the
recurrent layer (forward if 0, or backward if 1), and (l / 2) identifies the position
of the recurrent layer within the (forward or backward) stack.
Let op := getOperation(),
Then the ordering of parameter matrices is the list produced by concatenating
G(op, 0), G(op, 1), G(op, 2), ..., G(op, L-1).

For example:

Layout of elements within a weight matrix

Each parameter matrix is row-major in memory, and has the following dimensions:

Let K := { ::kUNIDIRECTION => 1
l := layer index (as described above)
E := getDataLength() (the embedding length)
isW := true if the matrix is an input (W) matrix, and false if
the matrix is a recurrent input (R) matrix.
if isW:
if l < K and ::kSKIP:
(numRows, numCols) := (0, 0) # input matrix is skipped
elif l < K and ::kLINEAR:
(numRows, numCols) := (H, E) # input matrix acts on input data size E
elif l >= K:
(numRows, numCols) := (H, K * H) # input matrix acts on previous hidden state
else: # not isW
(numRows, numCols) := (H, H)

In other words, the input weights of the first layer of the RNN (if not skipped) transform a getDataLength()-size column vector into a getHiddenSize()-size column vector. The input weights of subsequent layers transform a K*getHiddenSize()-size column vector into a getHiddenSize()-size column vector. K=2 in the bidirectional case to account for the full hidden state being the concatenation of the forward and backward RNN hidden states.

The recurrent weight matrices for all layers all have shape (H, H), both in the unidirectional and bidirectional cases. (In the bidirectional case, each recurrent weight matrix for the (forward or backward) RNN cell operates on the previous (forward or backward) RNN cell's hidden state, which is size H).

See also
getWeights(), RNNOperation

The documentation for this class was generated from the following file: