TRTEXEC with Mask Grounding DINO
The trtexec
tool is a command-line wrapper included as part of the TensorRT samples.
TAO 5.0.0 exposes the trtexec
tool in the TAO Deploy
container (or task group when run via launcher)
for deploying the model with an x86-based CPU and discrete GPUs. To run trtexec
on other platforms, such as
Jetson devices, or with versions of TensorRT that are not used by default in the TAO containers, you can follow the
official TensorRT documentation on how to get trtexec.
This section describes how to generate a TensorRT engine using trtexec
, which allows you to deploy TAO-trained
models on TensorRT, Triton, and Deepstream.
To generate an .onnx
file for Mask Grounding DINO, refer to the Mask Grounding DINO documentation.
The Mask Grounding DINO TensorRT engine only supports a batch size of 1.
trtexec --onnx=/path/to/model.onnx \
--minShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
--optShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
--maxShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
--fp16 \
--saveEngine=/path/to/save/trt/model.engine