RNN¶

Warning

This operator was deprecated in TensorRT 8.0 and will be removed in TensorRT 9.0; use the Loop operator instead.

Performs a recurrent layer computation such as Recurrent Neural Network (RNN), Gated Recurrent Network (GRU), and Long Short-Term Memory (LSTM).

Attributes¶

num_layers The layer count of the RNN.

hidden_size The hidden size of the RNN.

max_seq_length The maximum sequence length of the RNN.

data_length The length of the data being processed by the RNN for use in computing other values.

seq_lengths The sequence lengths in the batch. All sequence lengths should be in range \(1 \leq seq_len \leq max_seq_length\). The default value is max_seq_length.

operation The operation of the layer. Should be one of:

RELU Single gate RNN with ReLU activation
TANH : Single gate RNN with Tanh activation
LSTM : Four-gate LSTM network without peephole connections
GRU : Three-gate network consisting of GRUs

input_mode The input mode of the RNN. Should be one of:

LINEAR Perform the normal matrix multiplication in the first recurrent layer
SKIP No operation is performed on the first recurrent layer

direction The direction of the RNN. Should be one of:

UNIDIRECTION Network iterates from first input to last input
BIDIRECTION Network iterates from first to last (and vice versa) and outputs are concatenated

hidden_state The initial hidden state of the RNN. Should have the dimensions \([N_1, …, N_p, L, H]\), where \([N_1..N_p]\) are the index dimensions specified by the input tensor, L is the number of layers in the RNN, equal to num_layers, and H is the hidden state for each layer, equal to hidden_size if the direction is UNIDIRECTION, and \(2 \cdot hidden_size\) otherwise.

cell_state The initial cell state of the LSTM. Should have the dimensions \([N_1, …, N_p, L, H]\), where \([N_1..N_p]\) are the index dimensions specified by the input tensor, L is the number of layers in the RNN, equal to num_layers, H is the hidden state for each layer, equal to hidden_size if the direction is UNIDIRECTION, and \(2 \cdot hidden_size\) otherwise. It is an error to set this on an RNN layer that is not configured with an operation other than LSTM.

Inputs¶

input: tensor of type T.

Outputs¶

output: tensor of type T.

Data Types¶

T: float16, float32

Examples¶

RNN

in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 2, 2))
weights = np.array([[-6.3, 4.7, 2.9, 0.48]], dtype=np.dtype("f4"))
bias = np.array([[0.3, 0.09]], dtype=np.dtype("f4"))
layer = network.add_rnn_v2(in1, layer_count=1, hidden_size=2, max_seq_length=2, op=trt.RNNOperation.LSTM)
layer.set_weights_for_gate(0, trt.RNNGateType.INPUT, True, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.INPUT, True, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.FORGET, True, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.FORGET, True, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.CELL, True, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.CELL, True, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.OUTPUT, True, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.OUTPUT, True, trt.Weights(bias))

layer.set_weights_for_gate(0, trt.RNNGateType.INPUT, False, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.INPUT, False, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.FORGET, False, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.FORGET, False, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.CELL, False, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.CELL, False, trt.Weights(bias))
layer.set_weights_for_gate(0, trt.RNNGateType.OUTPUT, False, trt.Weights(weights))
layer.set_bias_for_gate(0, trt.RNNGateType.OUTPUT, False, trt.Weights(bias))


network.mark_output(layer.get_output(0))

inputs[in1.name] = np.array(
    [
        [
            [-3.0, -2.0],
            [0.0, 1.0]
        ],
    ])

outputs[layer.get_output(0).name] = layer.get_output(0).shape

expected[layer.get_output(0).name] = np.array(
    [
        [0.7616, 0.0],
        [0.449, 0.695]
    ])

C++ API¶

For more information about the C++ IRNNLayer operator, refer to the C++ IRNNLayer documentation.

Python API¶

For more information about the Python IRNNLayer operator, refer to the Python IRNNLayer documentation.