Mask2former with TAO Deploy#

To generate an optimized TensorRT engine, a Mask2former .onnx file, which is first generated using tao model mask2former export, is taken as an input to tao deploy mask2former gen_trt_engine. For more information about training a Mask2former model, refer to the Mask2former training documentation.

Each task is explained in detail in the following sections.

Note

Throughout this documentation, you will see references to $EXPERIMENT_ID and $DATASET_ID in the FTMS Client sections.
- For instructions on creating a dataset using the remote client, see the Creating a dataset section in the Remote Client documentation.
- For instructions on creating an experiment using the remote client, see the Creating an experiment section in the Remote Client documentation.
The spec format is YAML for TAO Launcher and JSON for FTMS Client.
File-related parameters, such as dataset paths or pretrained model paths, are required only for TAO Launcher and not for FTMS Client.

Converting .onnx File into TensorRT Engine#

To convert the .onnx file, you can reuse the spec file from the Exporting the model section.

gen_trt_engine#

The gen_trt_engine parameter defines TensorRT engine generation.

SPECS=$(tao-client mask2former get-spec --action gen_trt_engine --id $EXPERIMENT_ID)

gen_trt_engine:
  onnx_file: /path/to/onnx_file
  trt_engine: /path/to/trt_engine
  input_channel: 3
  input_width: 960
  input_height: 544
  tensorrt:
    data_type: fp16
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 10
    max_batch_size: 10
    calibration:
      cal_image_dir:
        - /path/to/cal/images
      cal_cache_file: /path/to/cal.bin
      cal_batch_size: 10
      cal_batches: 1000

Parameter	Datatype	Default	Description	Supported Values
`onnx_file`	string		The precision to be used for the TensorRT engine
`trt_engine`	string		The maximum workspace size for the TensorRT engine
`input_channel`	unsigned int	3	The input channel size. Only the value 3 is supported.	3
`input_width`	unsigned int	960	The input width	>0
`input_height`	unsigned int	544	The input height	>0
`batch_size`	unsigned int	-1	The batch size of the ONNX model	>=-1

tensorrt#

The tensorrt parameter defines TensorRT engine generation.

Parameter	Datatype	Default	Description	Supported Values
`data_type`	string	fp32	The precision to be used for the TensorRT engine	fp32/fp16
`workspace_size`	unsigned int	1024	The maximum workspace size for the TensorRT engine	>1024
`min_batch_size`	unsigned int	1	The minimum batch size used for the optimization profile shape	>0
`opt_batch_size`	unsigned int	1	The optimal batch size used for the optimization profile shape	>0
`max_batch_size`	unsigned int	1	The maximum batch size used for the optimization profile shape	>0

Use the following command to run Mask2former engine generation:

GEN_TRT_ENGINE_JOB_ID=$(tao-client mask2former experiment-run-action --action gen_trt_engine --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $EXPORT_JOB_ID)

Note

$EXPORT_JOB_ID is the job ID of the Exporting the model section.

tao deploy mask2former gen_trt_engine -e /path/to/spec.yaml \
         results_dir=/path/to/results \
         gen_trt_engine.onnx_file=/path/to/onnx/file \
         gen_trt_engine.trt_engine=/path/to/engine/file \
         gen_trt_engine.tensorrt.data_type=<data_type>

Required Arguments

-e, --experiment_spec: The experiment spec file to set up TensorRT engine generation

Optional Arguments

results_dir: The directory where the JSON status-log file will be dumped
gen_trt_engine.onnx_file: The .onnx model to be converted
gen_trt_engine.trt_engine: The path where the generated engine will be stored
gen_trt_engine.tensorrt.data_type: The precision to be exported

Sample Usage

Here’s an example of using the gen_trt_engine command to generate an FP16 TensorRT engine:

tao deploy mask2former gen_trt_engine -e $DEFAULT_SPEC
            gen_trt_engine.onnx_file=$ONNX_FILE \
            gen_trt_engine.trt_engine=$ENGINE_FILE \
            gen_trt_engine.tensorrt.data_type=FP16

Running Evaluation through a TensorRT Engine#

You can reuse the TAO evaluation spec file for evaluation through a TensorRT engine. The following is a sample spec file:

SPECS=$(tao-client mask2former get-spec --action evaluate --id $EXPERIMENT_ID)

evaluate:
  trt_engine: /path/to/engine/file
data:
  type: 'coco_panoptic'
  val:
    name: "coco_2017_val_panoptic"
    panoptic_json: "/datasets/coco/annotations/panoptic_val2017.json"
    img_dir: "/datasets/coco/val2017"
    panoptic_dir: "/datasets/coco/panoptic_val2017"
    batch_size: 1
    num_workers: 2

Use the following command to run Mask2former engine evaluation:

SPECS=$(tao-client mask2former experiment-run-action --action evaluate --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $GEN_TRT_ENGINE_JOB_ID)

tao deploy mask2former evaluate -e /path/to/spec.yaml: results_dir=/path/to/results evaluate.trt_engine=/path/to/engine/file

Required Arguments

-e, --experiment_spec: The experiment spec file for evaluation This should be the same as the tao evaluate spec file

Optional Arguments

results_dir: The directory where the JSON status-log file and evaluation results will be dumped
evaluate.trt_engine: The engine file for evaluation

Sample Usage

Here’s an example of using the evaluate command to run evaluation with a TensorRT engine:

tao deploy mask2former evaluate -e $DEFAULT_SPEC
            results_dir=$RESULTS_DIR \
            evaluate.trt_engine=$ENGINE_FILE

Running Inference through a TensorRT Engine#

You can reuse the TAO inference spec file for inference through a TensorRT engine. The following is a sample spec file:

SPECS=$(tao-client mask2former get-spec --action inference --id $EXPERIMENT_ID)

inference:
  trt_engine: /path/to/engine/file
  color_map: /path/to/colors.yaml
  label_map: /path/to/labels.csv
data:
  type: 'coco_panoptic'
  test:
    img_dir: /path/to/test_images/
    batch_size: 1

Use the following command to run Mask2former engine inference:

SPECS=$(tao-client mask2former experiment-run-action --action inference --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $GEN_TRT_ENGINE_JOB_ID)

tao deploy mask2former inference -e /path/to/spec.yaml: results_dir=/path/to/results inference.trt_engine=/path/to/engine/file

Required Arguments

-e, --experiment_spec: The experiment spec file for inference. This should be the same as the tao inference spec file.

Optional Arguments

results_dir: The directory where JSON status-log file and inference results will be dumped
inference.trt_engine: The engine file for inference

Sample Usage

Here’s an example of using the inference command to run inference with a TensorRT engine:

tao deploy mask2former inference -e $DEFAULT_SPEC
            results_dir=$RESULTS_DIR \
            evaluate.trt_engine=$ENGINE_FILE

The visualization will be stored in $RESULTS_DIR/images_annotated, and the COCO format predictions will be stored under $RESULTS_DIR/labels.