Grounding DINO with TAO Deploy#

To generate an optimized TensorRT engine for Grounding DINO, the gen_trt_engine action takes an ONNX file previously produced by the Grounding DINO export action. For more information about training a Grounding DINO model, refer to the Grounding DINO training documentation.

Converting ONNX File into TensorRT Engine#

You can reuse the spec from the Grounding DINO export action as a starting point.

gen_trt_engine#

The gen_trt_engine parameter defines TensorRT engine generation.

gen_trt_engine:
  onnx_file: /path/to/onnx_file
  trt_engine: /path/to/trt_engine
  input_channel: 3
  input_width: 960
  input_height: 544
  tensorrt:
    data_type: fp16
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 10
    max_batch_size: 10

Field

value_type

Description

default_value

valid_min

valid_max

valid_options

automl_enabled

results_dir

string

Path to where all the assets generated from a task are stored.

FALSE

gpu_id

int

The index of the GPU to build the TensorRT engine.

0

FALSE

onnx_file

string

Path to the ONNX model file.

???

FALSE
trt_engine
string
Path where the generated TensorRT engine from gen_trt_engine is stored.
This only works with tao-deploy.








FALSE

input_channel

int

Number of channels in the input tensor.

3

3

FALSE

input_width

int

Width of the input image tensor.

960

32

FALSE

input_height

int

Height of the input image tensor.

544

32

FALSE
opset_version
int
Operator set version of the ONNX model used to generate
the TensorRT engine.
17
1




FALSE
batch_size
int
The batch size of the input tensor for the engine.
A value of -1 implies dynamic tensor shapes.
-1
-1




FALSE

verbose

bool

Flag to enable verbose TensorRT logging.

False

FALSE

tensorrt

collection

Hyper parameters to configure the TensorRT Engine builder.

FALSE

tensorrt#

The tensorrt parameter defines the TensorRT engine generation.

Field

value_type

Description

default_value

valid_min

valid_max

valid_options

automl_enabled

data_type

string

The precision to be set for building the TensorRT engine.

FP32

FP32,FP16

FALSE
workspace_size
int
The size (in MB) of the workspace TensorRT has
to run it’s optimization tactics and generate the
TensorRT engine.
1024









FALSE
min_batch_size
int
The minimum batch size in the optimization profile for
the input tensor of the TensorRT engine.
1






FALSE
opt_batch_size
int
The optimum batch size in the optimization profile for
the input tensor of the TensorRT engine.
1






FALSE
max_batch_size
int
The maximum batch size in the optimization profile for
the input tensor of the TensorRT engine.
1






FALSE

Ask the agent to run the gen_trt_engine action against your spec. For example:

Build an FP16 TensorRT engine for Grounding DINO from the exported ONNX
at ``s3://my-bucket/gdino/model.onnx`` using ``trt-spec.yaml``. Write the
engine to ``s3://my-bucket/gdino/model.engine``. Run on local Docker.

Running Evaluation through a TensorRT Engine#

You can reuse the TAO evaluation specification file for evaluation through a TensorRT engine. The following is a sample specification file:

evaluate:
  trt_engine: /path/to/engine/file
  conf_threshold: 0.0
  input_width: 960
  input_height: 544
dataset:
  test_data_sources:
    image_dir: /data/raw-data/val2017/
    json_file: /data/raw-data/annotations/instances_val2017.json
  max_labels: 80
  batch_size: 8

Ask the agent to run the evaluate action against the engine you built. For example:

Evaluate the Grounding DINO TensorRT engine at
``s3://my-bucket/gdino/model.engine`` against ``eval-spec.yaml``. Run on
local Docker.

Running Inference through a TensorRT Engine#

You can reuse the TAO inference specification file for inference through a TensorRT engine. The following is a sample specification file:

inference:
  conf_threshold: 0.5
  input_width: 960
  input_height: 544
  trt_engine: /path/to/engine/file
  color_map:
    "black cat": green
    car: red
    person: blue
dataset:
  infer_data_sources:
    - image_dir: /path/to/coco/images/val2017/
      captions: ["black cat", "car", "person"]
  max_labels: 80
  batch_size: 8

Ask the agent to run the inference action against the engine you built. For example:

Run Grounding DINO inference with the TensorRT engine at
``s3://my-bucket/gdino/model.engine`` using ``infer-spec.yaml``. Run on
your chosen backend.

Annotated visualizations are written to images_annotated under the configured results directory, and KITTI-format predictions are written to labels.