Grounding DINO with TAO Deploy
To generate an optimized TensorRT engine, a Grounding DINO .onnx file, which is first generated using tao model grounding_dino export,
is taken as an input to tao deploy grounding_dino gen_trt_engine. For more information about training a Grounding DINO model,
refer to the Grounding DINO training documentation.
To convert the .onnx file, you can reuse the spec file from the tao model grounding_dino export command.
gen_trt_engine
The gen_trt_engine parameter defines TensorRT engine generation.
gen_trt_engine:
onnx_file: /path/to/onnx_file
trt_engine: /path/to/trt_engine
input_channel: 3
input_width: 960
input_height: 544
tensorrt:
data_type: fp16
workspace_size: 1024
min_batch_size: 1
opt_batch_size: 10
max_batch_size: 10
Field | value_type | Description | default_value | valid_min | valid_max | valid_options | automl_enabled |
|---|---|---|---|---|---|---|---|
results_dir |
string | Path to where all the assets generated from a task are stored. | FALSE | ||||
gpu_id |
int | The index of the GPU to build the TensorRT engine. | 0 | FALSE | |||
onnx_file |
string | Path to the ONNX model file. | ??? | FALSE | |||
|
|
string |
Path where the generated TensorRT engine from |
|
|
|
|
FALSE |
input_channel |
int | Number of channels in the input tensor. | 3 | 3 | FALSE | ||
input_width |
int | Width of the input image tensor. | 960 | 32 | FALSE | ||
input_height |
int | Height of the input image tensor. | 544 | 32 | FALSE | ||
|
|
int |
Operator set version of the ONNX model used to generate |
17 |
1 |
|
|
FALSE |
|
|
int |
The batch size of the input tensor for the engine. |
-1 |
-1 |
|
|
FALSE |
verbose |
bool | Flag to enable verbose TensorRT logging. | False | FALSE | |||
tensorrt |
collection | Hyper parameters to configure the TensorRT Engine builder. | FALSE |
tensorrt
The tensorrt parameter defines the TensorRT engine generation.
Field | value_type | Description | default_value | valid_min | valid_max | valid_options | automl_enabled |
|---|---|---|---|---|---|---|---|
data_type |
string | The precision to be set for building the TensorRT engine. | FP32 | FP32,FP16 | FALSE | ||
|
|
int |
The size (in MB) of the workspace TensorRT has |
1024 |
|
|
|
FALSE |
|
|
int |
The minimum batch size in the optimization profile for |
1 |
|
|
|
FALSE |
|
|
int |
The optimum batch size in the optimization profile for |
1 |
|
|
|
FALSE |
|
|
int |
The maximum batch size in the optimization profile for |
1 |
|
|
|
FALSE |
Use the following command to run Grounding DINO engine generation:
tao deploy grounding_dino gen_trt_engine -e /path/to/spec.yaml \
gen_trt_engine.onnx_file=/path/to/onnx/file \
gen_trt_engine.trt_engine=/path/to/engine/file \
gen_trt_engine.tensorrt.data_type=<data_type>
Required Arguments
-e, --experiment_spec: The experiment spec file to set up TensorRT engine generation.
Optional Arguments
gen_trt_engine.onnx_file: The.onnxmodel to be convertedgen_trt_engine.trt_engine: The path where the generated engine will be storedgen_trt_engine.tensorrt.data_type: The precision to be exported
Sample Usage
Here’s an example of using the gen_trt_engine command to generate an FP16 TensorRT engine:
tao deploy grounding_dino gen_trt_engine -e $DEFAULT_SPEC
gen_trt_engine.onnx_file=$ONNX_FILE \
gen_trt_engine.trt_engine=$ENGINE_FILE \
gen_trt_engine.tensorrt.data_type=FP16
You can reuse the TAO evaluation spec file for evaluation through a TensorRT engine. The following is a sample spec file:
evaluate:
trt_engine: /path/to/engine/file
conf_threshold: 0.0
input_width: 960
input_height: 544
dataset:
test_data_sources:
image_dir: /data/raw-data/val2017/
json_file: /data/raw-data/annotations/instances_val2017.json
max_labels: 80
batch_size: 8
Use the following command to run Grounding DINO engine evaluation:
tao deploy grounding_dino evaluate -e /path/to/spec.yaml \
evaluate.trt_engine=/path/to/engine/file
Required Arguments
-e, --experiment_spec: The experiment spec file for evaluation. This should be the same as thetao evaluatespec file.
Optional Arguments
evaluate.trt_engine: The engine file for evaluation.
Sample Usage
The following is an example of using the evaluate command to run evaluation with a TensorRT engine:
tao deploy grounding_dino evaluate -e $DEFAULT_SPEC
evaluate.trt_engine=$ENGINE_FILE
You can reuse the TAO inference spec file for inference through a TensorRT engine. The following is a sample spec file:
inference:
conf_threshold: 0.5
input_width: 960
input_height: 544
trt_engine: /path/to/engine/file
color_map:
"blackcat": green
car: red
person: blue
dataset:
infer_data_sources:
- image_dir: /path/to/coco/images/val2017/
captions: ["blackcat", "car", "person"]
max_labels: 80
batch_size: 8
Use the following command to run Grounding DINO engine inference:
tao deploy grounding_dino inference -e /path/to/spec.yaml \
inference.trt_engine=/path/to/engine/file
Required Arguments
-e, --experiment_spec: The experiment spec file for inference. This should be the same as thetao inferencespec file.
Optional Arguments
inference.trt_engine: The engine file for inference.
Sample Usage
The following is an example of using the inference command to run inference with a TensorRT engine:
tao deploy grounding_dino inference -e $DEFAULT_SPEC
results_dir=$RESULTS_DIR \
evaluate.trt_engine=$ENGINE_FILE
The visualization is stored in $RESULTS_DIR/images_annotated, and the KITTI format predictions is stored
under $RESULTS_DIR/labels.