DetectNet_v2 with TAO Deploy

To generate an optimized TensorRT engine for DetectNet_v2, the gen_trt_engine action takes an ONNX file previously produced by the DetectNet_v2 export action. For more information about training the DetectNet_v2, refer to the DetectNet_v2 training documentation.

Converting .onnx File into TensorRT Engine

DetectNet_v2 uses data from the training set for INT8 calibration. The data batches are sampled randomly across the entire training dataset, improving the accuracy of the INT8 model. Data pre-processing in the INT8 calibration step is the same as in the training process. Calibration occurs as a one-step process with the data batches generated on the fly. You can reuse the spec from the DetectNet_v2 export action as a starting point.

Note

When generating a TensorRT engine for a model trained with QAT enabled, the tensor scale factors defined by the calibration cache file are required. However, the current version of QAT does not natively support DLA int8 deployment on Jetson. To deploy this model on a Jetson with DLA int8, force post-training quantization to generate the calibration cache file.

Ask the agent to run the gen_trt_engine action against your spec. For example:

Build an INT8 TensorRT engine for DetectNet_v2 from the exported ONNX at
``s3://my-bucket/dnv2/model.onnx`` using ``trt-spec.yaml``. Calibrate
against the training data and write the engine to
``s3://my-bucket/dnv2/int8.engine``. Run on the local Docker backend.

Running Evaluation through TensorRT Engine

Use the same specification file as the TAO evaluation specification file. The following is a sample specification file:

dataset_config {
  validation_data_sources: {
      image_directory_path: "/workspace/tao-experiments/data/val/images"
      label_directory_path: "/workspace/tao-experiments/data/val/labels"
  }
  image_extension: "png"
  target_class_mapping {
      key: "car"
      value: "car"
  }
  target_class_mapping {
      key: "pedestrian"
      value: "pedestrian"
  }
  target_class_mapping {
      key: "cyclist"
      value: "cyclist"
  }
  target_class_mapping {
      key: "van"
      value: "car"
  }
  target_class_mapping {
      key: "person_sitting"
      value: "pedestrian"
  }
  validation_fold: 0
}
postprocessing_config {
  target_class_config {
      key: "car"
      value {
          clustering_config {
              clustering_algorithm: DBSCAN
              dbscan_confidence_threshold: 0.9
              coverage_threshold: 0.00499999988824
              dbscan_eps: 0.20000000298
              dbscan_min_samples: 0.0500000007451
              minimum_bounding_box_height: 20
          }
      }
  }
  target_class_config {
      key: "cyclist"
      value {
          clustering_config {
              clustering_algorithm: DBSCAN
              dbscan_confidence_threshold: 0.9
              coverage_threshold: 0.00499999988824
              dbscan_eps: 0.15000000596
              dbscan_min_samples: 0.0500000007451
              minimum_bounding_box_height: 20
          }
      }
  }
  target_class_config {
      key: "pedestrian"
      value {
          clustering_config {
              clustering_algorithm: DBSCAN
              dbscan_confidence_threshold: 0.9
              coverage_threshold: 0.00749999983236
              dbscan_eps: 0.230000004172
              dbscan_min_samples: 0.0500000007451
              minimum_bounding_box_height: 20
          }
      }
  }
}
evaluation_config {
  minimum_detection_ground_truth_overlap {
      key: "car"
      value: 0.699999988079
  }
  minimum_detection_ground_truth_overlap {
      key: "cyclist"
      value: 0.5
  }
  minimum_detection_ground_truth_overlap {
      key: "pedestrian"
      value: 0.5
  }
  evaluation_box_config {
      key: "car"
      value {
      minimum_height: 20
      maximum_height: 9999
      minimum_width: 10
      maximum_width: 9999
      }
  }
  evaluation_box_config {
      key: "cyclist"
      value {
      minimum_height: 20
      maximum_height: 9999
      minimum_width: 10
      maximum_width: 9999
      }
  }
  evaluation_box_config {
      key: "pedestrian"
      value {
      minimum_height: 20
      maximum_height: 9999
      minimum_width: 10
      maximum_width: 9999
      }
  }
}

Ask the agent to run the evaluate action against the engine you built. For example:

Evaluate the DetectNet_v2 TensorRT engine at
``s3://my-bucket/dnv2/int8.engine`` against ``eval-spec.yaml``. Run on
local Docker.

Running Inference through TensorRT Engine

Ask the agent to run the inference action against the engine you built. For example:

Run DetectNet_v2 inference with the TensorRT engine at
``s3://my-bucket/dnv2/int8.engine`` using ``infer-spec.yaml``. Run on
your chosen backend.

Annotated visualizations are written to images_annotated under the configured results directory, and KITTI-format predictions are written to labels.