Dequantize¶
Dequantize an integer input tensor into an float output tensor. The quantization computation is as follows: \(output_{i_0,..,i_n} = (input_{i_0,..,i_n}-\text{zero_point}) \cdot \text{scale}\).
Attributes¶
axis The axis to perform the quantization on.
toType The DataType of the output tensor. Defaults to float32.
Inputs¶
input: tensor of type T1.
scale: tensor of type T3 that provides the quantization scale. The scale tensor must be a build-time constant. Its dimensions must be a scalar for per-tensor quantization, a 1-D tensor for per-channel quantization, or a 2-D tensor for block quantization (supported for DataType::kINT4 only).
zero_point: tensor of type T2 that provides the quantization zero-point. The zero_point tensor is optional and will be assumed to be zero if not set. The zero_point must only contain zero-valued coefficients if set, and must have the same shape as scale.
Outputs¶
output: tensor of type T3.
Shape Information¶
input and output are tensors with a shape of \([a_0,...,a_n]\).
scale and zero_point must have the same shape, if zero_point is defined.
Volume Limits¶
input, scale, and zero_point can have up to \(2^{31}-1\) elements.
Examples¶
Dequantize
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 1, 3, 3))
scale = network.add_constant(shape=(1,), weights=np.array([1 / 127], dtype=np.float32))
quantize = network.add_quantize(in1, scale.get_output(0))
quantize.axis = 3
dequantize = network.add_dequantize(quantize.get_output(0), scale.get_output(0))
dequantize.axis = 3
network.mark_output(dequantize.get_output(0))
inputs[in1.name] = np.array(
[
[
[0.56, 0.89, 1.4],
[-0.56, 0.39, 6.0],
[0.67, 0.11, -3.6],
]
]
)
outputs[dequantize.get_output(0).name] = dequantize.get_output(0).shape
expected[dequantize.get_output(0).name] = np.array(
[
[
[0.56, 0.89, 1],
[-0.56, 0.39, 1.0],
[0.67, 0.11, -1.0],
]
]
)
Block Quantization
in1 = network.add_input("input1", dtype=trt.float32, shape=(1, 8))
weights = network.add_constant(shape=(4, 8), weights=np.array([
[1.0, 1.0, 2.0, 2.0, 3.0, 3.0, 4.0, 4.0],
[1.1, 1.2, 2.1, 2.2, 3.1, 3.2, 4.1, 4.2],
[4.0, 4.0, 5.0, 5.0, 6.0, 6.0, 7.0, 7.0],
[4.1, 4.2, 5.1, 5.2, 6.1, 6.2, 7.1, 7.2],
], dtype=np.float32))
scale = network.add_constant(shape=(2, 8), weights=np.array([
[1, 1, 2, 2, 3, 3, 4, 4],
[4, 4, 5, 5, 6, 6, 7, 7]
], dtype=np.float32))
quantize = network.add_quantize(weights.get_output(0), scale.get_output(0), trt.int4)
dequantize = network.add_dequantize(quantize.get_output(0), scale.get_output(0), trt.float32)
network.mark_output(dequantize.get_output(0))
inputs[in1.name] = np.array(
[
[2, 2, 2, 2, 2, 2, 2, 2],
]
)
outputs[dequantize.get_output(0).name] = dequantize.get_output(0).shape
expected[dequantize.get_output(0).name] = np.array(
[
[
[1, 1, 2, 2, 3, 3, 4, 4],
[1, 1, 2, 2, 3, 3, 4, 4],
[4, 4, 5, 5, 6, 6, 7, 7],
[4, 4, 5, 5, 6, 6, 7, 7],
]
]
)
C++ API¶
For more information about the C++ IDequantizeLayer operator, refer to the C++ IDequantizeLayer documentation.
Python API¶
For more information about the Python IDequantizeLayer operator, refer to the Python IDequantizeLayer documentation.