TRTEXEC with YOLO_v4_tiny
trtexec tool is a command-line wrapper included as part of the TensorRT samples.
TAO 5.0.0 exposes the
trtexec tool in the
TAO Deploy container (or task group when run via launcher)
for deploying the model with an x86-based CPU and discrete GPUs. To run
trtexec on other platforms, such as
Jetson devices, or with versions of TensorRT that are not used by default in the TAO containers, you can follow the
official TensorRT documentation on how to get trtexec.
This section describes how to generate a TensorRT engine using
trtexec, which allows you to deploy TAO-trained
models on TensorRT, Triton, and Deepstream.
To generate an
.onnx file for YOLO_v4_tiny, refer to the YOLO_v4_tiny documentation.
You can also refer to the YOLO_v4_tiny TAO-Deploy documentation
for instructions on generating an INT8 calibration file.
trtexec --onnx=/path/to/model.onnx \ --maxShapes=Input:16x3x384x1248 \ --minShapes=Input:1x3x384x1248 \ --optShapes=Input:8x3x384x1248 \ --calib=/path/to/int8/calib.txt \ --fp16 \ --int8 \ --saveEngine=/path/to/save/trt/model.engine