Accelerating TensorFlow 1.14.0 With TensorRT 6.0.1 Using The 19.09 Container

These release notes are for accelerating TensorFlow 1.14.0 with TensorRT version 6.0.1 using the TensorFlow 19.09NGC container. For specific details about TensorRT, see the TensorRT 6.0.1 Release Notes.

Key Features And Enhancements

This release includes the following key features and enhancements.

Compatibility

Limitations Of Accelerating TensorFlow With TensorRT

  • TF-TRT is not supported in the TensorRT containers.

Deprecated Features

  • The old API of TF-TRT is deprecated. It still works in TensorFlow 1.14, however, it may be removed in TensorFlow 2.0. The old API is a Python function named create_inference_graph which is not replaced by the Python class TrtGraphConverter with a number of methods. Refer to TF-TRT User Guide for more information about the API and how to use it.

Known Issues

  • Precision mode in the TF-TRT API is a string with one of the following values: FP32, FP16 or INT8. In TensorFlow 1.13, these strings were supported in lowercase, however, in TensorFlow 1.14 only uppercase is supported.
  • INT8 calibration (see the TF-TRT User Guide for more information about how to use INT8) is a very slow process that can take 1 hour depending on the model. We are working on optimizing this algorithm in TensorRT.
  • The pip package of TensorFlow 1.14 released by Google is missing TensorRT. This will be fixed in the next release of TensorFlow by Google. In the meantime, you can use the NVIDIA container for TensorFlow.