Installing TensorRT-Cloud CLI

Prerequisites

1. Install Linux or Windows. macOS is not supported.

2. Install Python 3.8 or above.

3. For refit support through CLI, the tensorrt or tensorrt-lean Python package must be installed with the version matching the TensorRT engine. The tensorrt-lean package is required for refitting version-compatible engines (those built with the --vc trtexec flag), and tensorrt is required for refitting non-version-compatible engines. Currently, we support TensorRT 10.0.1.

   python3 -m pip install tensorrt==10.0.1
   python3 -m pip install tensorrt-lean==10.0.1
   

Steps

The TensorRT-Cloud CLI tool is distributed as a Python wheel, packaged in a tar file. To get access to the tar, refer to Getting TensorRT-Cloud Access.

1. Retrieve the tar file containing the wheel and extract it.

   tar -xvf <NAME OF tar>.tar
   

2. Use pip to install.

   python3 -m pip install trt_cloud-0.1.0-py3-none-any.whl
   

3. Verify the TensorRT-Cloud CLI tool was installed successfully.

   trt-cloud --version
   

4. Get an overview of the CLI usage.

   trt-cloud --help