TRTEXEC with Mask Grounding DINO

The trtexec tool is a command-line wrapper included as part of the TensorRT samples. Refer to the official TensorRT documentation for instructions on installing or building trtexec on your target platform (including x86 with discrete GPU and Jetson devices).

This section describes how to generate a TensorRT engine using trtexec, which allows you to deploy TAO-trained models on TensorRT, Triton, and Deepstream.

Sample Command for a Mask Grounding DINO Model

To generate an .onnx file for Mask Grounding DINO, refer to the Mask Grounding DINO documentation.

Note

The Mask Grounding DINO TensorRT engine only supports a batch size of 1.

trtexec --onnx=/path/to/model.onnx \
        --minShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
        --optShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
        --maxShapes=inputs:1x3x544x960,input_ids:1x256,attention_mask:1x256,position_ids:1x256,token_type_ids:1x256,text_token_mask:1x256x256 \
        --fp16 \
        --saveEngine=/path/to/save/trt/model.engine