VisualChangeNet-Segmentation with TAO Deploy

To generate an optimized TensorRT engine for VisualChangeNet-Segmentation, the gen_trt_engine action takes an ONNX file previously produced by the VisualChangeNet export action. For more information about training a VisualChangeNet model, refer to the VisualChangeNet training documentation.

Converting an ONNX File into TensorRT Engine

gen_trt_engine

The following is an example configuration file for generating the TensorRT engine. Set task to segment.

task: segment
gen_trt_engine:
  results_dir: "${results_dir}/gen_trt_engine"
  onnx_file: "${results_dir}/export/changenet_model.onnx"
  trt_engine: "${results_dir}/gen_trt_engine/changenet.trt"
  input_channel: 3
  input_width: 128
  input_height: 512
  tensorrt:
    data_type: fp32
    workspace_size: int = 1024
    min_batch_size: int = 1
    opt_batch_size: int = 1
    max_batch_size: int = 1

The task section defines the change detection task for which the .onnx model was generated.

Parameter	Data Type	Default	Description
task	str	classify	A flag to indicate the change detection task. Currently supports two tasks: ‘segment’ and ‘classify’ for segmentation and classification

The gen_trt_engine section in the experiment specification file provides options for generating a TensorRT engine from an .onnx file.

Parameter	Datatype	Default	Description	Supported Values
results_dir	string	–	The path to the results directory	–
onnx_file	string	–	The path to the exported ETLT or ONNX model	–
trt_engine	string	–	The absolute path to the generated TensorRT engine	–
input_channel	unsigned int	3	The input channel size. Only a value of 3 is supported.	3
input_width	unsigned int	256	The input width	>0
input_height	unsigned int	256	The input height	>0
batch_size	unsigned int	-1	The batch size of the ONNX model	>=-1

tensorrt

The tensorrt parameter defines TensorRT engine generation.

Parameter	Datatype	Default	Description	Supported Values
data_type	string	fp32	The precision to be used for the TensorRT engine	fp32/fp16
workspace_size	unsigned int	1024	The maximum workspace size for the TensorRT engine	>1024
min_batch_size	unsigned int	1	The minimum batch size used for the optimization profile shape	>0
opt_batch_size	unsigned int	1	The optimal batch size used for the optimization profile shape	>0
max_batch_size	unsigned int	1	The maximum batch size used for the optimization profile shape	>0

Ask the agent to run the gen_trt_engine action against your spec. For example:

Build an FP32 TensorRT engine for VisualChangeNet-Segmentation from the
exported ONNX at ``s3://my-bucket/changenet/changenet_model.onnx`` using
``trt-spec.yaml``. Write the engine to
``s3://my-bucket/changenet/changenet.trt``. Run on local Docker.

Running Inference through TensorRT Engine

You can reuse the specification file that was specified for TAO inference. Set task to segment. The following is an example inference spec:

task: segment
dataset:
  segment:
    dataset: "CNDataset"
    root_dir: /path/to/root/dataset/dir/
    data_name: "LEVIR-CD"
    label_transform: "norm"
    batch_size: 16
    workers: 2
    num_classes: 2
    img_size: 256
    image_folder_name: "A"
    change_image_folder_name: "B"
    list_folder_name: 'list'
    annotation_folder_name: "label"
    test_split: "test"
    predict_split: 'predict'
    label_suffix: .png
inference:
  gpu_id: 0
  trt_engine: /path/to/engine/file
  results_dir: "${results_dir}/inference"

Ask the agent to run the inference action against the engine you built. For example:

Run VisualChangeNet-Segmentation inference with the TensorRT engine at
``s3://my-bucket/changenet/changenet.trt`` using ``infer-spec.yaml``.
Run on your chosen backend.

Visualizations are written to trt_inference under the configured results directory.

Running Evaluation through a TensorRT Engine

You can reuse the specification file that was specified for TAO evaluation through a TensorRT engine. Set task to segment. The following is a sample specification file:

task: segment
dataset:
  segment:
    dataset: "CNDataset"
    root_dir: /path/to/root/dataset/dir/
    data_name: "LEVIR-CD"
    label_transform: "norm"
    batch_size: 16
    workers: 2
    num_classes: 2
    img_size: 256
    image_folder_name: "A"
    change_image_folder_name: "B"
    list_folder_name: 'list'
    annotation_folder_name: "label"
    test_split: "test"
    predict_split: 'predict'
    label_suffix: .png
evaluate:
  gpu_id: 0
  trt_engine: /path/to/engine/file
  results_dir: "${results_dir}/inference"

Ask the agent to run the evaluate action against the engine you built. For example:

Evaluate the VisualChangeNet-Segmentation TensorRT engine at
``s3://my-bucket/changenet/changenet.trt`` against ``eval-spec.yaml``.
Run on local Docker.

Visualizations are written to trt_evaluate under the configured results directory.