PointPillars with TAO Deploy

To generate an optimized TensorRT engine for PointPillars, the gen_trt_engine action takes an ONNX file previously produced by the PointPillars export action.

Converting an .onnx File into TensorRT Engine

You can reuse the spec from the PointPillars export action as a starting point.

Ask the agent to run the gen_trt_engine action against your spec. For example:

Build an FP16 TensorRT engine for PointPillars from the exported ONNX
at ``s3://my-bucket/pointpillars/model.onnx`` using
``trt-spec.yaml``. Write the engine to
``s3://my-bucket/pointpillars/model.engine``. Run on local Docker.

Running Evaluation through TensorRT Engine

Use the same specification file as the TAO evaluation specification file. The following is a sample specification file:

dataset:
  class_names: ['Car', 'Pedestrian', 'Cyclist']
  type: 'GeneralPCDataset'
  data_path: '/media/data/zhimengf/tao-experiments/data/pointpillars'
  data_split: {
      'train': train,
      'test': val
  }
  data_info_path: "/media/data/zhimengf/tao-experiments/pointpillars/data_info"
  info_path: {
      'train': [infos_train.pkl],
      'test': [infos_val.pkl],
  }
  balanced_resampling: False
  point_feature_encoding: {
      encoding_type: absolute_coordinates_encoding,
      used_feature_list: ['x', 'y', 'z', 'intensity'],
      src_feature_list: ['x', 'y', 'z', 'intensity'],
  }
  point_cloud_range: [0, -39.68, -3, 69.12, 39.68, 1]
  data_augmentor:
    disable_aug_list: ['placeholder']
    aug_config_list:
      - name: gt_sampling
        db_info_path:
            - dbinfos_train.pkl
        preface: {
           filter_by_min_points: ['Car:5', 'Pedestrian:5', 'Cyclist:5'],
        }
        sample_groups: ['Car:15','Pedestrian:15', 'Cyclist:15']
        num_point_features: 4
        disable_with_fake_lidar: False
        remove_extra_width: [0.0, 0.0, 0.0]
        limit_whole_scene: False
      - name: random_world_flip
        along_axis_list: ['x']
      - name: random_world_rotation
        world_rot_angle: [-0.78539816, 0.78539816]
      - name: random_world_scaling
        world_scale_range: [0.95, 1.05]
  data_processor:
    - name: mask_points_and_boxes_outside_range
      remove_outside_boxes: True
  num_workers: 4

model:
  post_processing:
    recall_thresh_list: [0.3, 0.5, 0.7]
    score_thresh: 0.1
    output_raw_score: False
    eval_metric: kitti
    nms_config:
      multi_classes_nms: False
      nms_type: nms_gpu
      nms_thresh: 0.01
      nms_pre_max_size: 4096
      nms_post_max_size: 500

evaluate:
  batch_size: 1
  trt_engine: "/media/data/zhimengf/tao-experiments/pointpillars/retrain/checkpoint_epoch_80.onnx.fp16"
  results_dir: "/data/zhimengf/tao-experiments/tao-deploy/evaluate"

Ask the agent to run the evaluate action against the engine you built. For example:

Evaluate the PointPillars TensorRT engine at
``s3://my-bucket/pointpillars/model.engine`` against ``eval-spec.yaml``.
Run on local Docker.

Running Inference through TensorRT Engine

Ask the agent to run the inference action against the engine you built. For example:

Run PointPillars inference with the TensorRT engine at
``s3://my-bucket/pointpillars/model.engine`` using ``infer-spec.yaml``.
Run on your chosen backend.

Visualizations are written to results_dir as configured in the spec.