TensorRT 4.0 Python API Documentation¶

Installation¶

First, ensure that you have TensorRT 4.0 installed and accessible in your LD_LIBRARY_PATH. You can find it here.

Installing the Python APIs¶

You can install the Python API using the wheels located in the python folder.

Python 2.7: pip2 install tensorrt-*_16.04-cp27-cp27mu-linux_x86_64.whl
Python 3.5: pip3 install tensorrt-*_16.04-cp35-cp35m-linux_x86_64.whl

Where tensorrt-* is the version you are installing (e.g. tensorrt-4.0.1.3).

Installing the Graph Surgeon¶

TensorRT includes the Graph Surgeon, which can transform TensorFlow graphs to ensure compatibility with the UFF converter. It can be used to specify plugin nodes for operations that might be unsupported in the UFF converter or TensorRT, and is required to handle certain TensorFlow ops. You can install it using the wheel located in the graphsurgeon folder.

pip install graphsurgeon-0.2.0-py2.py3-none-any.whl

Installing the UFF Toolkit¶

TensorRT includes the Universal Framework Format (UFF) toolkit, which can convert models trained in TensorFlow to a format from which TensorRT can build engines. You can install it using the wheel located in the uff folder. Note that the UFF converter is only designed and tested for x86 platforms. To use TensorRT with UFF on other platforms, first generate the UFF file on an x86 platform.

pip install uff-0.4.0-py2.py3-none-any.whl

Installing PyCUDA¶

TensorRT uses PyCUDA to interface with the GPU. You can install PyCUDA from here.

Use Cases and Examples¶

Running an existing TensorFlow model with TensorRT.
- Converting Tensorflow to TensorRT
Running an existing Caffe model with TensorRT.
- Converting a Caffe model to TensorRT
Manually constructing a TensorRT engine (for use with PyTorch, for example).
- Manually constructing a TensorRT engine

Deploying a TensorRT engine as part of a larger application, such as a web backend.

We have included a few sample applications with the TensorRT Python package in the examples directory.