DINO with TAO Deploy

To generate an optimized TensorRT engine, a DINO .onnx file, which is first generated using tao model dino export, is taken as an input to tao deploy dino gen_trt_engine. For more information about training a DINO model, refer to the DINO training documentation.

To convert the .onnx file, you can reuse the spec file from the tao model dino export command.

gen_trt_engine

The gen_trt_engine parameter defines TensorRT engine generation.

            

            
gen_trt_engine:
  onnx_file: /path/to/onnx_file
  trt_engine: /path/to/trt_engine
  input_channel: 3
  input_width: 960
  input_height: 544
  tensorrt:
    data_type: int8
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 10
    max_batch_size: 10
    calibration:
      cal_image_dir:
        - /path/to/cal/images
      cal_cache_file: /path/to/cal.bin
      cal_batch_size: 10
      cal_batches: 1000
        

tensorrt

The tensorrt parameter defines TensorRT engine generation.

calibration

The calibration parameter defines TensorRT engine generation with PTQ INT8 calibration.

Use the following command to run DINO engine generation:

            

            
tao deploy dino gen_trt_engine -e /path/to/spec.yaml \
           -r /path/to/results \
           gen_trt_engine.onnx_file=/path/to/onnx/file \
           gen_trt_engine.trt_engine=/path/to/engine/file \
           gen_trt_engine.tensorrt.data_type=<data_type>
        

Required Arguments

• -e, --experiment_spec: The experiment spec file to set up TensorRT engine generation

Optional Arguments

• -r, --results_dir: The directory where the JSON status-log file will be dumped

• gen_trt_engine.onnx_file: The .onnx model to be converted

• gen_trt_engine.trt_engine: The path where the generated engine will be stored

• gen_trt_engine.tensorrt.data_type: The precision to be exported

Sample Usage

Here’s an example of using the gen_trt_engine command to generate an FP16 TensorRT engine:

            

            
tao deploy dino gen_trt_engine -e $DEFAULT_SPEC
           gen_trt_engine.onnx_file=$ONNX_FILE \
           gen_trt_engine.trt_engine=$ENGINE_FILE \
           gen_trt_engine.tensorrt.data_type=FP16
        

You can reuse the TAO evaluation spec file for evaluation through a TensorRT engine. The following is a sample spec file:

            

            
evaluate:
  trt_engine: /path/to/engine/file
  conf_threshold: 0.0
  input_width: 960
  input_height: 544
dataset:
  test_data_sources:
    image_dir: /data/raw-data/val2017/
    json_file: /data/raw-data/annotations/instances_val2017.json
  num_classes: 91
  batch_size: 8
        

Use the following command to run DINO engine evaluation:

            

            
tao deploy dino evaluate -e /path/to/spec.yaml \
           -r /path/to/results \
           evaluate.trt_engine=/path/to/engine/file
        

Required Arguments

• -e, --experiment_spec: The experiment spec file for evaluation This should be the same as the tao evaluate spec file

Optional Arguments

• -r, --results_dir: The directory where the JSON status-log file and evaluation results will be dumped

• evaluate.trt_engine: The engine file for evaluation

Sample Usage

Here’s an example of using the evaluate command to run evaluation with a TensorRT engine:

            

            
tao deploy dino evaluate -e $DEFAULT_SPEC
           -r $RESULTS_DIR \
           evaluate.trt_engine=$ENGINE_FILE
        

You can reuse the TAO inference spec file for inference through a TensorRT engine. The following is a sample spec file:

            

            
inference:
  conf_threshold: 0.5
  input_width: 960
  input_height: 544
  trt_engine: /path/to/engine/file
  color_map:
    person: green
    car: red
    cat: blue
dataset:
  infer_data_sources:
    image_dir: /data/raw-data/val2017/
    classmap: /path/to/coco/annotations/coco_classmap.txt
  num_classes: 91
  batch_size: 8
        

Use the following command to run DINO engine inference:

            

            
tao deploy dino inference -e /path/to/spec.yaml \
           -r /path/to/results \
           inference.trt_engine=/path/to/engine/file
        

Required Arguments

• -e, --experiment_spec: The experiment spec file for inference. This should be the same as the tao inference spec file.

Optional Arguments

• -r, --results_dir: The directory where JSON status-log file and inference results will be dumped

• inference.trt_engine: The engine file for inference

Sample Usage

Here’s an example of using the inference command to run inference with a TensorRT engine:

            

            
tao deploy dino inference -e $DEFAULT_SPEC
           -r $RESULTS_DIR \
           evaluate.trt_engine=$ENGINE_FILE
        

The visualization will be stored in $RESULTS_DIR/images_annotated, and the KITTI format predictions will be stored under $RESULTS_DIR/labels.