TensorRT
6.0.1.5
|
A RNN layer in a network definition. More...
#include <NvInfer.h>
Public Member Functions | |
virtual unsigned | getLayerCount () const =0 |
Get the number of layers in the RNN. More... | |
virtual std::size_t | getHiddenSize () const =0 |
Get the size of the hidden layers. More... | |
virtual int | getSeqLength () const =0 |
Get the sequence length. More... | |
virtual void | setOperation (RNNOperation op)=0 |
Set the operation of the RNN layer. More... | |
virtual RNNOperation | getOperation () const =0 |
Get the operation of the RNN layer. More... | |
virtual void | setInputMode (RNNInputMode op)=0 |
Set the operation of the RNN layer. More... | |
virtual RNNInputMode | getInputMode () const =0 |
Get the operation of the RNN layer. More... | |
virtual void | setDirection (RNNDirection op)=0 |
Set the direction of the RNN layer. More... | |
virtual RNNDirection | getDirection () const =0 |
Get the direction of the RNN layer. More... | |
virtual void | setWeights (Weights weights)=0 |
Set the weight parameters for the RNN. More... | |
virtual Weights | getWeights () const =0 |
Get the W weights for the RNN. More... | |
virtual void | setBias (Weights bias)=0 |
Set the bias parameters for the RNN. More... | |
virtual Weights | getBias () const =0 |
Get the bias parameter vector for the RNN. More... | |
virtual int | getDataLength () const =0 |
Get the length of the data being processed by the RNN for use in computing other values. More... | |
virtual void | setHiddenState (ITensor &hidden)=0 |
Set the initial hidden state of the RNN with the provided hidden ITensor. More... | |
virtual ITensor * | getHiddenState () const =0 |
Get the initial hidden state of the RNN. More... | |
virtual void | setCellState (ITensor &cell)=0 |
Set the initial cell state of the RNN with the provided cell ITensor. More... | |
virtual ITensor * | getCellState () const =0 |
Get the initial cell state of the RNN. More... | |
Public Member Functions inherited from nvinfer1::ILayer | |
virtual LayerType | getType () const =0 |
Return the type of a layer. More... | |
virtual void | setName (const char *name)=0 |
Set the name of a layer. More... | |
virtual const char * | getName () const =0 |
Return the name of a layer. More... | |
virtual int | getNbInputs () const =0 |
Get the number of inputs of a layer. | |
virtual ITensor * | getInput (int index) const =0 |
Get the layer input corresponding to the given index. More... | |
virtual int | getNbOutputs () const =0 |
Get the number of outputs of a layer. | |
virtual ITensor * | getOutput (int index) const =0 |
Get the layer output corresponding to the given index. More... | |
virtual void | setInput (int index, ITensor &tensor)=0 |
replace an input of this layer with a specific tensor More... | |
virtual void | setPrecision (DataType dataType)=0 |
Set the computational precision of this layer. More... | |
virtual DataType | getPrecision () const =0 |
get the computational precision of this layer More... | |
virtual bool | precisionIsSet () const =0 |
whether the computational precision has been set for this layer More... | |
virtual void | resetPrecision ()=0 |
reset the computational precision for this layer More... | |
virtual void | setOutputType (int index, DataType dataType)=0 |
Set the output type of this layer. More... | |
virtual DataType | getOutputType (int index) const =0 |
get the output type of this layer More... | |
virtual bool | outputTypeIsSet (int index) const =0 |
whether the output type has been set for this layer More... | |
virtual void | resetOutputType (int index)=0 |
reset the output type for this layer More... | |
A RNN layer in a network definition.
This layer applies an RNN operation on the inputs.
|
pure virtual |
Get the bias parameter vector for the RNN.
|
pure virtual |
Get the initial cell state of the RNN.
|
pure virtual |
Get the length of the data being processed by the RNN for use in computing other values.
|
pure virtual |
Get the direction of the RNN layer.
|
pure virtual |
Get the size of the hidden layers.
The hidden size is the value of hiddenSize parameter passed into addRNN().
|
pure virtual |
Get the initial hidden state of the RNN.
|
pure virtual |
Get the operation of the RNN layer.
|
pure virtual |
Get the number of layers in the RNN.
|
pure virtual |
Get the operation of the RNN layer.
|
pure virtual |
Get the sequence length.
The sequence length is the maximum number of time steps passed into the addRNN() function. This is also the maximum number of input tensors that the RNN can process at once.
|
pure virtual |
Get the W weights for the RNN.
|
pure virtual |
Set the bias parameters for the RNN.
bias | The weight structure holding the bias parameters. |
The trained weights for the bias parameter vectors of the RNN. The DataType for this structure must be kFLOAT or kHALF, and must be the same datatype as the input tensor.
The layout of the weight structure depends on the RNNOperation, RNNInputMode, and RNNDirection of the layer. The array specified by weights.values
contains a sequence of bias vectors, where each bias vector is linearly appended after the previous without padding; e.g., if bias vector 0 and 1 have M and N elements respectively, then the layout of weights.values
in memory looks like:
The ordering of bias vectors is similar to the ordering of weight matrices as described in setWeights(). To determine the order of bias vectors for a given RNN configuration, determine the ordered list of weight matrices [ W0, W1, ..., Wn ]
. Then replace each weight matrix with its corresponding bias vector, i.e. apply the following transform (for layer l
, gate g
):
Wl[g]
becomes Wbl[g]
Rl[g]
becomes Rbl[g]
For example:
an RNN with getLayerCount() == 3
, getDirection() == kUNIDIRECTION
, and getOperation() == kRELU
has the following order:
[ Wb0[i], Rb0[i], Wb1[i], Rb1[i], Wb2[i], Rb2[i] ]
an RNN with getLayerCount() == 2
, getDirection() == kUNIDIRECTION
, and getOperation() == kGRU
has the following order:
[ Wb0[z], Wb0[r], Wb0[h], Rb0[z], Rb0[r], Rb0[h], Wb1[z], Wb1[r], Wb1[h], Rb1[z], Rb1[r], Rb1[h] ]
an RNN with getLayerCount() == 2
, getDirection() == kBIDIRECTION
, and getOperation() == kRELU
has the following order:
[ Wb0_fw[i], Rb0_fw[i], Wb0_bw[i], Rb0_bw[i], Wb1_fw[i], Rb1_fw[i], Wb1_bw[i], Rb1_bw[i] ]
(fw = "forward", bw = "backward")
Each bias vector has a fixed size, getHiddenSize().
|
pure virtual |
Set the initial cell state of the RNN with the provided cell
ITensor.
cell | The initial cell state of the RNN. |
The layout for cell
is a linear layout of a 3D matrix:
If cell
is not specified, then the initial cell state is set to zero.
If getDirection() is kBIDIRECTION, the amount of space required is doubled and C is equal to getLayerCount() * 2.
The cell state only affects LSTM RNN's.
|
pure virtual |
Set the direction of the RNN layer.
The direction determines if the RNN is run as a unidirectional(left to right) or bidirectional(left to right and right to left). In the kBIDIRECTION case the output is concatenated together, resulting in output size of 2x getHiddenSize().
|
pure virtual |
Set the initial hidden state of the RNN with the provided hidden
ITensor.
hidden | The initial hidden state of the RNN. |
The layout for hidden
is a linear layout of a 3D matrix:
If getDirection() is kBIDIRECTION, the amount of space required is doubled and C is equal to getLayerCount() * 2.
If hidden is not specified, then the initial hidden state is set to zero.
|
pure virtual |
Set the operation of the RNN layer.
|
pure virtual |
Set the operation of the RNN layer.
|
pure virtual |
Set the weight parameters for the RNN.
weights | The weight structure holding the weight parameters. |
The trained weights for the weight parameter matrices of the RNN. The DataType for this structure must be kFLOAT or kHALF, and must be the same datatype as the input tensor.
The layout of the weight structure depends on the RNNOperation, RNNInputMode, and RNNDirection of the layer. The array specified by weights.values
contains a sequence of parameter matrices, where each parameter matrix is linearly appended after the previous without padding; e.g., if parameter matrix 0 and 1 have M and N elements respectively, then the layout of weights.values
in memory looks like:
The following sections describe the order of weight matrices and the layout of elements within a weight matrix.
The parameter matrices are ordered as described below:
For example:
an RNN with getLayerCount() == 3
, getDirection() == kUNIDIRECTION
, and getOperation() == kRELU
has the following order:
[ W0[i], R0[i], W1[i], R1[i], W2[i], R2[i] ]
an RNN with getLayerCount() == 2
, getDirection() == kUNIDIRECTION
, and getOperation() == kGRU
has the following order:
[ W0[z], W0[r], W0[h], R0[z], R0[r], R0[h], W1[z], W1[r], W1[h], R1[z], R1[r], R1[h] ]
an RNN with getLayerCount() == 2
, getDirection() == kBIDIRECTION
, and getOperation() == kRELU
has the following order:
[ W0_fw[i], R0_fw[i], W0_bw[i], R0_bw[i], W1_fw[i], R1_fw[i], W1_bw[i], R1_bw[i] ]
(fw = "forward", bw = "backward")
Each parameter matrix is row-major in memory, and has the following dimensions:
In other words, the input weights of the first layer of the RNN (if not skipped) transform a getDataLength()
-size column vector into a getHiddenSize()
-size column vector. The input weights of subsequent layers transform a K*getHiddenSize()
-size column vector into a getHiddenSize()
-size column vector. K=2
in the bidirectional case to account for the full hidden state being the concatenation of the forward and backward RNN hidden states.
The recurrent weight matrices for all layers all have shape (H, H)
, both in the unidirectional and bidirectional cases. (In the bidirectional case, each recurrent weight matrix for the (forward or backward) RNN cell operates on the previous (forward or backward) RNN cell's hidden state, which is size H
).