This code snippet demonstrates the how the DNN module with safe DLA enabled is typically used. Note that error handling is left out for clarity.
Initialize network from file.
In order to be able to use safe DLA, the model must be generated using --useSafeDLA
option via TensorRT Optimizer Tool tool. The processor type, while initializing DNN, must be either DW_PROCESSOR_TYPE_DLA_0
or DW_PROCESSOR_TYPE_DLA_1
depending on which DLA engine the inference should take place.
Check that the loaded network has the expected number of inputs and outputs.
Ask the DNN about the order of the input and output blobs. The network is assumed to contain the input blob "data_in" and output blobs "data_out1" and "data_out2".
Note that, safe DLA requires RGBA
input with interleaved channels, and it provides outputs with a NCHWx
format.
NCHWx
format's layout is equivalent to a C array with dimensions [N][(C+x-1)/x][H][W][x]
, with the tensor coordinates (n, c, h, w)
mapping to array subscript [n][c/x][h][w][cx]
where:
N
: Batch size n
: Batch index C
: Number of planes c
: Plane index H
: Height h
: Vertical index W
: Width w
: horizontal index x
: Number of interleaved elements
DLA dictates that x
is equal to 32 / sizeof(DataType)
; therefore, for a tensor with FP16 precision, x
is 16
.
Moreover, the input and output to a safe DLA model are expected to be tensors of type NvMedia. In order to simplify the process of inference, dwDataConditioner
and dwDNN
modules provide the streaming and conversion functionalities.
In dwDNNTensors, the dimensions for NCHWx are stored as:
Therefore, in order to compute the number of channels, which is needed for conversion to NCHW
:
Below, we shall make use of these features:
Convert DNN input from image to tensor, then perform DNN inference and stream results back. All operations are performed asynchronously with the host code.
Finally, free previously allocated memory.