TensorRT Release 18.02
The NVIDIA container image of TensorRT, release 18.02, is available.
TensorRT container image version 18.02 is based on TensorRT 3.0.4.
Contents of TensorRT
This container image contains an example deployment strategy using TensorRT inference exposed via a REST server. Three trained models, NVCaffe, ONNX and TensorFlow, are included to demonstrate the inference REST server. You can also perform inference using your own NVCaffe, ONNX and TensorFlow models via the REST server.
- The TensorRT documentation and samples. The samples can be built by running make in the /workspace/tensorrt/samples directory. The resulting executables are in the /workspace/tensorrt/bin directory.
- The example NVCaffe MNIST model and the caffe_mnist script are located in the /workspace/tensorrt_server directory. The script runs the REST server to provide inference for that model via an HTTP endpoint.
- The example Inception-v1 ONNX model and the onnx_inception_v1 script are also located in the /workspace/tensorrt_server directory. This example and script runs the REST server to provide inference for that model via an HTTP endpoint.
- The example ResNet-152 TensorFlow model and the tensorflow_resnet script are also located in the /workspace/tensorrt_server directory. This example and script runs the REST server to provide inference for that model via an HTTP endpoint.
- Ubuntu 16.04 including Python 2.7 environment
- NVIDIA CUDA 9.0.176 including:
- CUDA® Basic Linear Algebra Subroutines library™ (cuBLAS) 9.0.282 Patch 2 which is installed by default
- cuBLAS 9.0.234 Patch 1 as a debian file. Installing Patch 1 by issuing the dpkg -i /opt/cuda-cublas-9-0_9.0.234-1_amd64.deb command is the workaround for the known issue described below.
- NVIDIA CUDA® Deep Neural Network library™ (cuDNN) 7.0.5
- NCCL 2.1.2 (optimized for NVLink™ )
Key Features and Enhancements
Known Issues
cuBLAS 9.0.282 regresses RNN seq2seq FP16 performance for a small subset of input sizes. This issue should be fixed in the next update. As a workaround, install cuBLAS 9.0.234 Patch 1 by issuing the dpkg -i /opt/cuda-cublas-9-0_9.0.234-1_amd64.deb command.