OCRNet with TAO Deploy

An OCRNet .etlt or .onnx file generated from tao export is taken as an input to tao-deploy to generate an optimized TensorRT engine. For more information about training the OCRNet, please refer to OCRNet training documentation.

Each task is explained in detail in the following sections.

Note

• Throughout this documentation, you will see references to $EXPERIMENT_ID and $DATASET_ID in the FTMS Client sections.

  • For instructions on creating a dataset using the remote client, see the Creating a dataset section in the Remote Client documentation.

  • For instructions on creating an experiment using the remote client, see the Creating an experiment section in the Remote Client documentation.

• The spec format is YAML for TAO Launcher and JSON for FTMS Client.

• File-related parameters, such as dataset paths or pretrained model paths, are required only for TAO Launcher and not for FTMS Client.

Converting an .etlt or .onnx File into TensorRT Engine

gen_trt_engine

The gen_trt_engine parameter in the experiment specification file provides options to generate the TensorRT engine from .etlt` or .onnx.

gen_trt_engine:
    onnx_file: "??"
    results_dir: "${results_dir}/convert_dataset"

Parameter	Datatype	Default	Description	Supported Values
onnx_file	String	–	The absolute path to the exported .etlt or .onnx model	–
trt_engine	String	–	The absolute path to the generated TensorRT engine	–
gpu_id	Unsigned int	0	The GPU device index	Valid gpu index
input_channel	Unsigned int	1	The input channel of the TensorRT engine	>0
input_width	Unsigned int	100	The input width of the TensorRT engine	>0
input_height	Unsigned int	32	The input height of the TensorRT engine	>0
opset_version	Unsigned int	12	The ONNX opset version	Valid ONNX opset version
batch_size	Unsigned int	-1	The batch size of the TensorRT engine. Set it to -1 to enable dynamic batch.	-1 or >0
verbose	Bool	False	A flag to enable verbose information output during TensorRT engine generation	True/False
tensorrt	Dict config	–	Other options for TensorRT-engine generation	–
results_dir	String	–	The absolute path to the gen_trt_engine log output	–

tensorrt

The tensorrt parameter provides more options for TensorRT generation.

tensorrt:
    data_type: fp16
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 1
    max_batch_size: 1

Parameter	Datatype	Default	Description	Supported Values
data_type	String	fp16	The precision of the generated TensorRT engine	fp16,FP32
workspace_size	Unsigned int	1024	The workspace size of the generated TensorRT engine	>0
min_batch_size	Unsigned int	1	The minimum batch size of the generated TensorRT engine	>0
opt_batch_size	Unsigned int	1	The optimal batch size of the generated TensorRT engine	>0
max_batch_size	Unsigned int	1	The maximum batch size of the generated TensorRT engine	>0

Use the following command to generate the TensorRT engine:

FTMS ClientTAO Launcher

GEN_TRT_ENGINE_JOB_ID=$(tao-client ocrnet experiment-run-action --action gen_trt_engine --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $EXPORT_JOB_ID)

Running Evaluation through the TensorRT Engine

The evaluate parameter in the experiment specification file provides options to set evaluation with TensorRT engine:

evaluate:
    trt_engine: "??"
    test_dataset_dir: "/path/to/test_images_directory"
    test_dataset_gt_file: "/path/to/gt_file_list"
    input_width: 100
    input_height: 32

Parameter	Datatype	Default	Description	Supported Values
trt_engine	String	–	The absolute path to the TensorRT engine	–
gput_id	Unsigned int	0	The GPU device index	Valid gpu index
test_dataset_dir	String	–	The absolute path to the test images directory	–
test_dataset_gt_file	String	–	The absolute path to the ground truth file for test_images. The required format for the ground truth is described in the Preparing Data section.	>0
input_width	Unsigned int	100	The input width of the TensorRT engine	>0
input_height	Unsigned int	32	The input height of the TensorRT engine	>0
batch_size	Unsigned int	1	The batch size of the inference	>0
results_dir	String	–	The absolute path to the gen_trt_engine log output	–

Use the following command to run evaluation with the TensorRT engine:

FTMS ClientTAO Launcher

EVALUATE_JOB_ID=$(tao-client ocrnet experiment-run-action --action evaluate --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $GEN_TRT_ENGINE_JOB_ID)

Running Inference through TensorRT Engine

The inference parameter in the experiment specification file provides options to set evaluation with TensorRT engine:

inference:
    trt_engine: "??"
    inference_dataset_dir: "/path/to/test_images_directory"
    input_width: 100
    input_height: 32

Parameter	Datatype	Default	Description	Supported Values
trt_engine	String	–	The absolute path to the TensorRT engine	–
gput_id	Unsigned int	0	The GPU device index	Valid gpu index
inference_dataset_dir	String	–	The absolute path to the inference images directory	–
input_width	Unsigned int	100	The input width of the TensorRT engine	>0
input_height	Unsigned int	32	The input height of the TensorRT engine	>0
batch_size	Unsigned int	1	The batch size of the inference	>0
results_dir	String	–	The absolute path to the gen_trt_engine log output	–

Use the following command to run inference with the TensorRT engine:

FTMS ClientTAO Launcher

INFERENCE_JOB_ID=$(tao-client ocrnet experiment-run-action --action inference --id $EXPERIMENT_ID --specs "$SPECS" --parent_job_id $GEN_TRT_ENGINE_JOB_ID)