Installing TensorRT-Cloud CLI

Prerequisites

1. Install Linux, Windows, or MacOS.

2. Install Python 3.8 or above.

3. For refit support through CLI, the tensorrt or tensorrt-lean Python package must be installed with the version matching the TensorRT engine. The tensorrt-lean package is required for refitting version-compatible engines (those built with the --vc trtexec flag), and tensorrt is required for refitting non-version-compatible engines. Currently, we support TensorRT 10.0.1.

   python3 -m pip install tensorrt==10.0.1
   python3 -m pip install tensorrt-lean==10.0.1
   

Note

refit is not supported on MacOS as TensorRT does not support MacOS.

Steps

The TensorRT-Cloud CLI tool is distributed as a Python wheel, packaged in a tar file. To get access to the tar, refer to Getting TensorRT-Cloud Access.

1. Use pip to install.

   python3 -m pip install trt-cloud
   

2. Verify the TensorRT-Cloud CLI tool was installed successfully.

   trt-cloud --version
   

3. Get an overview of the CLI usage.

   trt-cloud --help