OCRNet with TAO Deploy

An OCRNet .etlt or .onnx file generated from tao export is taken as an input to tao-deploy to generate an optimized TensorRT engine. For more information about training the OCRNet, please refer to OCRNet training documentation.

Converting an .etlt or .onnx File into TensorRT Engine

gen_trt_engine

The gen_trt_engine parameter in the experiment specification file provides options to generate the TensorRT engine from .etlt` or .onnx.

Copy
Copied!

            
            gen_trt_engine:
    onnx_file: "??"
    results_dir: "${results_dir}/convert_dataset"

Parameter	Datatype	Default	Description	Supported Values
`onnx_file`	String	–	The absolute path to the exported `.etlt` or `.onnx` model	–
`trt_engine`	String	–	The absolute path to the generated TensorRT engine	–
`gpu_id`	Unsigned int	0	The GPU device index	Valid gpu index
`input_channel`	Unsigned int	1	The input channel of the TensorRT engine	>0
`input_width`	Unsigned int	100	The input width of the TensorRT engine	>0
`input_height`	Unsigned int	32	The input height of the TensorRT engine	>0
`opset_version`	Unsigned int	12	The ONNX opset version	Valid ONNX opset version
`batch_size`	Unsigned int	-1	The batch size of the TensorRT engine. Set it to `-1` to enable dynamic batch.	-1 or >0
`verbose`	Bool	False	A flag to enable verbose information output during TensorRT engine generation	True/False
`tensorrt`	Dict config	–	Other options for TensorRT-engine generation	–
`results_dir`	String	–	The absolute path to the `gen_trt_engine` log output	–

tensorrt

The tensorrt parameter provides more options for TensorRT generation.

Copy
Copied!

            
            tensorrt:
    data_type: fp16
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 1
    max_batch_size: 1

Parameter	Datatype	Default	Description	Supported Values
`data_type`	String	fp16	The precision of the generated TensorRT engine	fp16,FP32
`workspace_size`	Unsigned int	1024	The workspace size of the generated TensorRT engine	>0
`min_batch_size`	Unsigned int	1	The minimum batch size of the generated TensorRT engine	>0
`opt_batch_size`	Unsigned int	1	The optimal batch size of the generated TensorRT engine	>0
`max_batch_size`	Unsigned int	1	The maximum batch size of the generated TensorRT engine	>0

Use the following command to generate the TensorRT engine:

Copy
Copied!

            
            tao deploy ocrnet gen_trt_engine -e <experiment_spec_file>
                                 results_dir=<global_results_dir>
                                 [gen_trt_engine.<gen_trt_engine_option>=<gen_trt_engine_option_value>]

Required Arguments

-e, --experiment_spec_file: The path to the experiment spec file.
results_dir: The global results directory. The engine generation log will be saved in results_dir.

Optional Arguments

You can set optional arguments to override the option values in the experiment spec file:

gen_trt_engine.<gen_trt_engine_option>: The generate TensorRT engine options.

Here’s an example for using the OCRNet evaluate command:

Copy
Copied!

            
            tao deploy ocrnet gen_trt_engine -e $DEFAULT_SPEC \
                                 results_dir=$RESULTS_DIR \
                                 gen_trt_engine.onnx_file=$ONNX_TAO_MODEL \
                                 gen_trt_engine.trt_engine=$PATH_TO_SAVED_ENGINE

Running Evaluation through the TensorRT Engine

The evaluate parameter in the experiment specification file provides options to set evaluation with TensorRT engine:

Copy
Copied!

            
            evaluate:
    trt_engine: "??"
    test_dataset_dir: "/path/to/test_images_directory"
    test_dataset_gt_file: "/path/to/gt_file_list"
    input_width: 100
    input_height: 32

Parameter	Datatype	Default	Description	Supported Values
`trt_engine`	String	–	The absolute path to the TensorRT engine	–
`gput_id`	Unsigned int	0	The GPU device index	Valid gpu index
`test_dataset_dir`	String	–	The absolute path to the test images directory	–
`test_dataset_gt_file`	String	–	The absolute path to the ground truth file for `test_images`. The required format for the ground truth is described in the Preparing Data section.	>0
`input_width`	Unsigned int	100	The input width of the TensorRT engine	>0
`input_height`	Unsigned int	32	The input height of the TensorRT engine	>0
`batch_size`	Unsigned int	1	The batch size of the inference	>0
`results_dir`	String	–	The absolute path to the `gen_trt_engine` log output	–

Use the following command to run evaluation with the TensorRT engine:

Copy
Copied!

            
            tao deploy ocrnet evaluate -e <experiment_spec_file>
                           results_dir=<global_results_dir>
                           [evaluate.<evaluate_option>=<evaluate_value>]

Required Arguments

-e, --experiment_spec_file: The path to the experiment spec file.
results_dir: The global results directory. The engine generation log will be saved in results_dir.

Optional Arguments

You can set the optional arguments to override the options values in the experiment spec file.

evaluate.<evaluate_option>: The evaluate options.

Here’s an example of using the OCRNet evaluate command:

Copy
Copied!

            
            tao deploy ocrnet evaluate -e $DEFAULT_SPEC \
                           results_dir=$RESULTS_DIR \
                           evaluate.test_dataset_dir=$EVALUATE_IMG_DIR \
                           evaluate.test_dataset_gt_file=$EVALUATE_GT_FILE \
                           evaluate.trt_engine=$PATH_TO_SAVED_ENGINE

Running Inference through TensorRT Engine

The inference parameter in the experiment specification file provides options to set evaluation with TensorRT engine:

Copy
Copied!

            
            inference:
    trt_engine: "??"
    inference_dataset_dir: "/path/to/test_images_directory"
    input_width: 100
    input_height: 32

Parameter	Datatype	Default	Description	Supported Values
`trt_engine`	String	–	The absolute path to the TensorRT engine	–
`gput_id`	Unsigned int	0	The GPU device index	Valid gpu index
`inference_dataset_dir`	String	–	The absolute path to the inference images directory	–
`input_width`	Unsigned int	100	The input width of the TensorRT engine	>0
`input_height`	Unsigned int	32	The input height of the TensorRT engine	>0
`batch_size`	Unsigned int	1	The batch size of the inference	>0
`results_dir`	String	–	The absolute path to the gen_trt_engine log output	–

Use the following command to run inference with the TensorRT engine:

Copy
Copied!

            
            tao deploy ocrnet inference -e <experiment_spec_file>
                               results_dir=<global_results_dir>
                               [inference.<inference_option>=<evaluate_value>]

Required Arguments

-e, --experiment_spec_file: The path to the experiment spec file.
results_dir: The global results directory. The engine generation log will be saved in results_dir.

Optional Arguments

You can set the optional arguments to override the option values in the experiment spec file.

inference.<inference_option>: The inference options.

Here’s an example of using the OCRNet evaluate command:

Copy
Copied!

            
            tao deploy ocrnet inference -e $DEFAULT_SPEC \
                            results_dir=$RESULTS_DIR \
                            inference.inference_dataset_dir=$INFERENCE_IMAGES_DIR \
                            inference.trt_engine=$PATH_TO_SAVED_ENGINE