Classification (TF2) with TAO Deploy#

TF2 Classification ONNX file generated from tao export is taken as an input to tao deploy to generate optimized TensorRT engine. For more information about training the TF2 Classification, please refer to TF2 Classification training documentation.

Converting ONNX File into TensorRT Engine#

Same spec file can be used with the tao model classification_tf2 export command.

GenTrtEngine Config#

The gen_trt_engine configuration contains the parameters of exporting a .onnx model to TensorRT engine, which can be used for deployment.

Field

Description

Data Type and Constraints

Recommended/Typical Value

onnx_file

The path to the exported .onnx model

string

trt_engine

The path where the generated engine will be stored

string

results_dir

Directory to save the output log. If not specified log will be saved under global $results_dir/gen_trt_engine

string

tensorrt

TensorRT config

Dict

The tensorrt configuration contains specification of the TensorRT engine and calibration requirements. +——————————+———————————————————————-+——————————-+——————————-+ | Field | Description | Data Type and Constraints | Recommended/Typical Value | +——————————+———————————————————————-+——————————-+——————————-+ | data_type | The precision to be used for the TensorRT engine | string | FP32 | +——————————+———————————————————————-+——————————-+——————————-+ | min_batch_size | The minimum batch size used for optimization profile shape | unsigned int | 1 | +——————————+———————————————————————-+——————————-+——————————-+ | opt_batch_size | The optimal batch size used for optimization profile shape | unsigned int | 1 | +——————————+———————————————————————-+——————————-+——————————-+ | max_batch_size | The maximum batch size used for optimization profile shape | unsigned int | 1 | +——————————+———————————————————————-+——————————-+——————————-+ | max_workspace_size | The maximum workspace size for the TensorRT engine | unsigned int | 2 | +——————————+———————————————————————-+——————————-+——————————-+ | calibration | Calibration config | Dict | | +——————————+———————————————————————-+——————————-+——————————-+

The calibration configuration specifies the location of the calibration data and where to save the calibration cache file. +——————————+———————————————————————-+——————————-+——————————-+ | Field | Description | Data Type and Constraints | Recommended/Typical Value | +——————————+———————————————————————-+——————————-+——————————-+ | cal_image_dir | The directory containing images to be used for calibration | string | | +——————————+———————————————————————-+——————————-+——————————-+ | cal_cache_file | The path to calibration cache file | string | | +——————————+———————————————————————-+——————————-+——————————-+ | cal_batches | The number of batches to be iterated for calibration | unsigned int | 10 | +——————————+———————————————————————-+——————————-+——————————-+ | cal_batch_size | The batch size for each batch | unsigned int | 1 | +——————————+———————————————————————-+——————————-+——————————-+ | cal_data_file | The path to calibration data file | string | | +——————————+———————————————————————-+——————————-+——————————-+

SPECS=$(tao-client classification_tf2 get-spec --action gen_trt_engine --job_type experiment --id $EXPERIMENT_ID)

See also

For information on how to create an experiment using the remote client, refer to the Creating an experiment section in the Remote Client documentation.

Use the following command to run TF2 Classification engine generation:

GEN_TRT_ENGINE_JOB_ID=$(tao-client classification_tf2 experiment-run-action --action gen_trt_engine --id $EXPERIMENT_ID --specs "$SPECS")

See also

For information on how to create an experiment using the remote client, refer to the Creating an experiment section in the Remote Client documentation.

Running Evaluation through TensorRT Engine#

Same spec file as TAO evaluation spec file.

Use the following command to run TF2 Classification engine evaluation:

EVAL_JOB_ID=$(tao-client classification_tf2 experiment-run-action --action evaluate --id $EXPERIMENT_ID --parent_job_id $GEN_TRT_ENGINE_JOB_ID --specs "$SPECS")

See also

For information on how to create an experiment using the remote client, refer to the Creating an experiment section in the Remote Client documentation.

Running Inference through TensorRT Engine#

Same spec file as TAO inference spec file.

Use the following command to run TF2 Classification engine inference:

INFERENCE_JOB_ID=$(tao-client classification_tf2 experiment-run-action --action inference --id $EXPERIMENT_ID --parent_job_id $GEN_TRT_ENGINE_JOB_ID --specs "$SPECS")

See also

For information on how to create an experiment using the remote client, refer to the Creating an experiment section in the Remote Client documentation.