VisualChangeNet-Classification with TAO Deploy#

To generate an optimized TensorRT engine for VisualChangeNet-Classification, the gen_trt_engine action takes an ONNX file previously produced by the VisualChangeNet export action. For more information about training a VisualChangeNet model, refer to the VisualChangeNet training documentation.

Converting an ONNX File into TensorRT Engine#

gen_trt_engine#

The following is an example configuration file for generating the TensorRT engine. Set task to classify.

task: classify
gen_trt_engine:
  results_dir: "${results_dir}/gen_trt_engine"
  onnx_file: "${results_dir}/export/changenet_model.onnx"
  trt_engine: "${results_dir}/gen_trt_engine/changenet.trt"
  input_channel: 3
  input_width: 128
  input_height: 512
  tensorrt:
    data_type: fp32
    workspace_size: int = 1024
    min_batch_size: int = 1
    opt_batch_size: int = 1
    max_batch_size: int = 1

The task section defines the change detection task for which the .onnx model was generated.

Parameter	Data Type	Default	Description
`task`	str	classify	A flag to indicate the change detection task. Supports two tasks: ‘segment’ and ‘classify’ for segmentation and classification.

The gen_trt_engine section in the experiment specification file provides options for generating a TensorRT engine from an .onnx file.

Parameter	Datatype	Default	Description	Supported Values
`results_dir`	string	–	The path to the results directory	–
`onnx_file`	string	–	The path to the exported ETLT or ONNX model	–
`trt_engine`	string	–	The absolute path to the generated TensorRT engine	–
`input_channel`	unsigned int	3	The input channel size. Only a value of 3 is supported.	3
`input_width`	unsigned int	128	The input width	>0
`input_height`	unsigned int	512	The input height	>0
`batch_size`	unsigned int	-1	The batch size of the ONNX model	>=-1

tensorrt#

The tensorrt parameter defines TensorRT engine generation.

Parameter	Datatype	Default	Description	Supported Values
`data_type`	string	fp32	The precision to be used for the TensorRT engine	fp32/fp16
`workspace_size`	unsigned int	1024	The maximum workspace size for the TensorRT engine	>1024
`min_batch_size`	unsigned int	1	The minimum batch size used for the optimization profile shape	>0
`opt_batch_size`	unsigned int	1	The optimal batch size used for the optimization profile shape	>0
`max_batch_size`	unsigned int	1	The maximum batch size used for the optimization profile shape	>0

Ask the agent to run the gen_trt_engine action against your spec. For example:

Build an FP32 TensorRT engine for VisualChangeNet-Classification from
the exported ONNX at ``s3://my-bucket/changenet/changenet_model.onnx``
using ``trt-spec.yaml``. Write the engine to
``s3://my-bucket/changenet/changenet.trt``. Run on the local Docker backend.

Running Inference through TensorRT Engine#

You can reuse the specification file that was specified for TAO inference. The following is an example inference spec:

task: classify
model:
  classify:
    eval_margin: 0.5
dataset:
  classify:
    infer_dataset:
      csv_path: /path/to/infer.csv
      images_dir: /path/to/img_dir
    image_ext: .jpg
    batch_size: 16
    workers: 2
    num_input: 4
    input_map:
      LowAngleLight: 0
      SolderLight: 1
      UniformLight: 2
      WhiteLight: 3
    concat_type: linear
    grid_map:
      x: 2
      y: 2
    output_shape:
      - 128
      - 128
    augmentation_config:
      rgb_input_mean: [0.485, 0.456, 0.406]
      rgb_input_std: [0.229, 0.224, 0.225]
    num_classes: 2
inference:
  gpu_id: 0
  trt_engine: /path/to/engine/file
  results_dir: "${results_dir}/inference"

Ask the agent to run the inference action against the engine you built. For example:

Run VisualChangeNet-Classification inference with the TensorRT engine
at ``s3://my-bucket/changenet/changenet.trt`` using ``infer-spec.yaml``.
Run on your chosen backend.

Running Evaluation through a TensorRT Engine#

You can reuse the specification file that was specified for TAO evaluation through a TensorRT engine. The following is a sample specification file:

task: classify
model:
  classify:
    eval_margin: 0.5
dataset:
  classify:
    infer_dataset:
      csv_path: /path/to/infer.csv
      images_dir: /path/to/img_dir
    image_ext: .jpg
    batch_size: 16
    workers: 2
    num_input: 4
    input_map:
      LowAngleLight: 0
      SolderLight: 1
      UniformLight: 2
      WhiteLight: 3
    concat_type: linear
    grid_map:
      x: 2
      y: 2
    output_shape:
      - 128
      - 128
    augmentation_config:
      rgb_input_mean: [0.485, 0.456, 0.406]
      rgb_input_std: [0.229, 0.224, 0.225]
    num_classes: 2
evaluate:
  gpu_id: 0
  trt_engine: /path/to/engine/file
  results_dir: "${results_dir}/inference"

Ask the agent to run the evaluate action against the engine you built. For example:

Evaluate the VisualChangeNet-Classification TensorRT engine at
``s3://my-bucket/changenet/changenet.trt`` against ``eval-spec.yaml``.
Run on local Docker.