This release adds support for the Jetson Xavier NX module, and includes new versions of CUDA, TensorRT, and cuDNN. Support for Vulkan 1.2 has been added as well.

In addition, DeepStream 5.0 Developer Preview is supported in this release.

L4T 32.4.2

• Vulkan 1.2 support

• Support for upgrading L4T version using Debian package management tool¹

• Support for Generic Timestamping Engine (GTE) for Jetson AGX Xavier and Jetson Xavier NX

  • Kernel driver for GTE

  • Sample GPIO driver using GTE for timestamping input state transitions

  Refer to the GTE section in the NVIDIA Jetson Linux Developer Guide.

• Support for Dynamic Frequency Scaling (DFS) for VIC using actmon.

• Samples to demonstrate hardware backed authentication and encryption capabilities of Jetson TX2, Jetson AGX Xavier and Jetson Xavier NX.

  Refer to the Security section in the NVIDIA Jetson Linux Developer Guide.

• Utility to burn fuse with multiple Jetson devices simultaneously.

• For Jetson Nano and Jetson Xavier NX Developer Kits:

  • Option to select APP partition size on the microSD card during initial configuration at first boot.

¹ Debian package-based L4T upgrade is supported only starting with L4T version 32.3.1.

CUDA 10.2

• Performance optimization through user-mode submits.

  50% launch latency reduction for CUDA kernels, resulting in improved GPU utilization and lower CPU utilization.

TensorRT 7.1.0 (Developer Preview)

• New Layers

  1. IFillLayer: The IFillLayer is used to generate an output tensor with the specified mode.

  2. IIteratorLayer: The IIteratorLayer enables a loop to iterate over a tensor. A loop is defined by loop boundary layers.

  3. ILoopBoundaryLayer: Class ILoopBoundaryLayer defines a virtual method getLoop() that returns a pointer to the associated ILoop.

  4. ILoopOutputLayer: The ILoopOutputLayer specifies an output from the loop.

  5. IParametricReluLayer: The IParametricReluLayer represents a parametric ReLU operation, meaning, a leaky ReLU where the slopes for x < 0 can be different for each element.

  6. IRecurrenceLayer: The IRecurrenceLayer specifies a recurrent definition.

  7. ISelectLayer: The ISelectLayer returns either of the two inputs depending on the condition.

  8. ITripLimitLayer: The ITripLimitLayer specifies how many times the loop iterates.

• New Operators

  Expanded support for ONNX operations: Added ConstantOfShape, DequantizeLinear, Equal, Erf, Expand, Greater, GRU, Less, Loop, LRN, LSTM, Not, PRelu, QuantizeLinear, RandomUniform, RandomUniformLike, Range, RNN, Scan, Sqrt, Tile, and Where.

• New Samples

  sampleAlgorithmSelector shows an example of how to use the algorithm selection API based on sampleMNIST. This sample demonstrates the usage of IAlgorithmSelector to deterministically build TensorRT engines.

• Working with Loops

  TensorRT supports loop-like constructs, which can be useful for recurrent networks. TensorRT loops support scanning over input tensors, recurrent definitions of tensors, and both "scan outputs" and "last value" outputs.

• ONNX parser with dynamic shapes support

  The ONNX parser supports full-dimensions mode only. Your network definition must be created with the explicitBatch flag set.

• BERT INT8 and mixed precision optimizations

  Some GEMM layers are now followed by GELU activation in the BERT model. Since TensorRT doesn't have IMMA GEMM layers, you can implement those GEMM layers in the BERT network with either IConvolutionLayer or IFullyConnectedLayer layers depending on what precision you require. For example, you can leverage IConvolutionLayer with H == W == 1 (CONV1x1) to implement a FullyConnected operation and leverage IMMA math under INT8 mode. TensorRT supports the fusion of Convolution/FullyConnected and GELU.

• Working with Quantized Networks

  Supports quantized models trained with Quantization Aware Training. Support is limited to symmetrically quantized models, meaning zero_point = 0 using QuantizeLinear and DequantizeLinear.

• Boolean Tensor Support

  TensorRT supports Boolean tensors which can be marked as network input and output. IElementWiseLayer, IUnaryLayer (only kNOT), IShuffleLayer, ITripLimit (only kWHILE) and ISelectLayer support the Boolean datatype. Boolean tensors can be used only with FP32 and FP16 precision networks.

• Working with empty tensor

  TensorRT supports empty tensors. A tensor is an empty tensor if it has one or more dimensions with length zero.

• Builder layer timing cache

  The layer timing cache will cache the layer profiling information during the builder phase. Models with repeated layer will see a significant speedup in builder time.

• Pointwise fusion based on code generation

  Pointwise fusion is updated to use code generation and runtime compilation to further improve performance.

• Dilation support for deconvolution

  IDeconvolutionLayer now supports a dilation parameter. This is accessible through the C++ API, Python API, and the ONNX parser.

• Selecting FP16 and INT8 kernels

  TensorRT supports Mixed Precision Inference with FP32, FP16, or INT8 as supported precisions. Depending on the hardware support, you can choose to enable either of the above precision to accelerate inference.

• Calibration with dynamic shapes

  INT8 calibration with dynamic shapes supports the same functionality as a standard INT8 calibrator but for networks with dynamic shapes.

• Algorithm selection

  Algorithm selection provides a mechanism to select and report algorithms for different layers in a network. This can also be used to deterministically build TensorRT engine or to reproduce the same implementations for layers in the engine.

  Refer to the release notes for TensorRT 7.X.X for detailed release notes.

cuDNN 8.0.0 (Developer Preview)

• cuDNN library split into multiple inferencing and training libraries, enabling applications to only link against needed cuDNN sub-components.

VPI 0.2.0 (Developer Preview)

• Performance optimization of algorithms introduced in VPI 0.1.0: up to 45x on GPU and up to 90x on CPU backends.

  Refer to the VPI documentation for benchmarks.

• New Image FFT, Image iFFT and Image Format converter algorithms added with support for CPU and GPU backends. (PVA backend will be supported in a future release.)

• NVIDIA Nsight Systems 2020.2 for application profiling across GPU and CPU.

  • Enhanced data analysis with option to export to SQLite, HDF5 or JSON

  • Support for sampling Xavier PMU extensions

  • Reduced NVTX overhead

  • New CLI support for profiling on devices with intermittent network connectivity

• NVIDIA Nsight Graphics 2020.1 for graphics application debugging and profiling.

  • Added support to save and load custom named layouts

  • Improved events view display and filtering

  • Enhanced support for mixed DPI monitor scaling

  • Added precise control of pixel position in the resources view for launching pixel history

  • Improved Vulkan action profiling information by fixing GPU clocks when running profiling experiments

  • Adds support for new Vulkan extensions:

    VK_EXT_line_rasterization

    VK_EXT_headless_surface

    VK_KHR_create_renderpass2

    VK_KHR_external_fence_win32

    VK_KHR_external_memory_win32

    VK_KHR_external_semaphore

    VK_KHR_imageless_framebuffer

    VK_NVX_image_view_handle

• NVIDIA Nsight Compute 2019.3 for CUDA kernel profiling.

  • NVIDIA Nsight Compute version remains the same from JetPack 4.3