NVIDIA Documentation Center

Deep Learning Performance

GPUs accelerate machine learning operations by performing calculations in parallel. Many operations, especially those representable as matrix multipliers will see good acceleration right out of the box. Even better performance can be achieved by tweaking operation parameters to efficiently use GPU resources. The performance documents present the tips that we think are most widely useful.

NVIDIA cuDNN

The NVIDIA CUDA Deep Neural Network (cuDNN) library is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. Deep learning researchers and framework developers worldwide rely on cuDNN for high-performance GPU acceleration.

NVIDIA DALI

The NVIDIA Data Loading Library (DALI) is a collection of highly optimized building blocks, and an execution engine, for accelerating the pre-processing of input data for deep learning applications. DALI provides both the performance and the flexibility for accelerating different data pipelines as a single library. This single library can then be easily integrated into different deep learning training and inference applications.

NVIDIA DIGITS

The NVIDIA Deep Learning GPU Training System (DIGITS) can be used to rapidly train highly accurate deep neural networks (DNNs) for image classification, segmentation, and object-detection tasks. DIGITS simplifies common deep learning tasks such as managing data, designing and training neural networks on multi-GPU systems, monitoring performance in real time with advanced visualizations, and selecting the best-performing model from the results browser for deployment.

NVIDIA Maxine

NVIDIA Maxine is a GPU-accelerated SDK with state-of-the-art AI features for developers to build virtual collaboration and content creation applications such as video conferencing and live streaming. Maxine’s AI SDKs, such as Video Effects, Audio Effects, and Augmented Reality (AR) are highly optimized and include modular features that can be chained into end-to-end pipelines to deliver the highest performance possible on GPUs, both on PCs and in data centers.

NVIDIA NeMo

NVIDIA Neural Modules (NeMo) is a flexible, Python-based toolkit enabling data scientists and researchers to build state-of-the-art speech and language deep learning models composed of reusable building blocks that can be safely connected together for conversational AI applications.

NVIDIA NCCL

The NVIDIA Collective Communications Library (NCCL) is a library of multi-GPU collective communication primitives that are topology-aware and can be easily integrated into applications. Collective communication algorithms employ many processors working in concert to aggregate data. NCCL is not a full-blown parallel programming framework; rather, it’s a library focused on accelerating collective communication primitives.

NVIDIA Optimized Frameworks

NVIDIA Optimized Frameworks such as Kaldi, NVIDIA Optimized Deep Learning Framework (powered by Apache MXNet), NVCaffe, PyTorch, and TensorFlow (which includes DLProf and TF-TRT) offer flexibility with designing and training custom (DNNs for machine learning and AI applications.

NVIDIA Riva Speech Skills

NVIDIA Riva is an SDK for building multimodal conversational systems. Riva is used for building and deploying AI applications that fuse vision, speech, sensors, and services together to achieve conversational AI use cases that are specific to a domain of expertise. It offers a complete workflow to build, train, and deploy AI systems that can use visual cues such as gestures and gaze along with speech in context.

NVIDIA TensorRT

NVIDIA TensorRT is an SDK for high-performance deep learning inference. It includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for deep learning inference applications. The core of NVIDIA TensorRT is a C++ library that facilitates high-performance inference on NVIDIA GPUs. TensorRT takes a trained network, which consists of a network definition and a set of trained parameters, and produces a highly optimized runtime engine that performs inference for that network.

NVIDIA Triton Inference Server

NVIDIA Triton Inference Server (formerly TensorRT Inference Server) provides a cloud inferencing solution optimized for NVIDIA GPUs. The server provides an inference service via an HTTP or gRPC endpoint, allowing remote clients to request inferencing for any model being managed by the server.

NVIDIA Clara Deploy Application Framework

Clara Deploy Application Framework provides a container-based development and deployment framework for building AI-accelerated medical imaging workflows. Using Kubernetes, a multi-staged, container-based pipeline can be defined. The modular architecture allows end-to-end use of the platform’s offerings or customization of the workflow pipelines with bring-your-own algorithms.

NVIDIA Clara Parabricks Pipelines

Clara Parabricks is a complete software solution for next-generation sequencing, including short- and long-read applications, supporting workflows that start with basecalling and extend through tertiary analysis. Clara Parabricks Pipelines were built to optimize acceleration, accuracy, and scalability. Users can achieve a 35-50X acceleration and 99.99 percent accuracy for variant calling when comparing against CPU-only BWA-GATK4 pipelines. It can run the full GATK4 Best Practices and is also fully configurable, letting users choose which steps, parameter settings, and versions of the pipeline to run.

NVIDIA Clara Parabricks Toolkit

The Clara Parabricks Toolkit is a technology stack of CUDA-accelerated libraries and deep learning modules, C++ and Python APIs, reference applications, and integrations with third-party applications and workflows for HPC, deep learning, and data analytics tools in genomics.

NVIDIA Clara Train Application Framework

Clara Train Application Framework is a domain-optimized, developer application framework that includes APIs for AI-assisted annotation, making any medical viewer AI-capable. It also includes a TensorFlow-based training framework with pre-trained models to kickstart AI development with techniques like transfer learning, federated learning and AutoML.

NVIDIA cuBLAS Library

The cuBLAS library is an implementation of Basic Linear Algebra Subprograms (BLAS) on the NVIDIA CUDA runtime. It enables the user to access the computational resources of NVIDIA GPUs.

NVIDIA cuFFT Library

The NVIDIA CUDA Fast Fourier Transform (cuFFT) library consists of two components: cuFFT and cuFFTW. The cuFFT library provides high performance on NVIDIA GPUs, and the cuFFTW library is a porting tool to use the Fastest Fourier Transform in the West (FFTW) on NVIDIA GPUs.

NVIDIA cuRAND Library

The NVIDIA CUDA Random Number Generation (cuRAND) library provides an API for simple and efficient generation of high-quality pseudorandom and quasirandom numbers.

NVIDIA cuSOLVER Library

The cuSOLVER library is a high-level package based on cuBLAS and cuSPARSE libraries. It provides Linear Algebra Package (LAPACK)-like features such as common matrix factorization and triangular solve routines for dense matrices.

NVIDIA cuSPARSE Library

The cuSPARSE library contains a set of basic linear algebra subroutines used for handling sparse matrices. It’s implemented on the NVIDIA CUDA runtime and is designed to be called from C and C++.

NVIDIA cuSPARSELt Library

The cuSPARSELt library provides high-performance, structured, matrix-dense matrix multiplication functionality. cuSPARSELt allows users to exploit the computational resources of the latest NVIDIA GPUs.

NVIDIA cuTENSOR Library

The cuTENSOR library is a first-of-its-kind, GPU-accelerated tensor linear algebra library, providing high-performance tensor contraction, reduction, and element-wise operations. cuTENSOR is used to accelerate applications in the areas of deep learning training and inference, computer vision, quantum chemistry, and computational physics.

NVIDIA NPP Library

NVIDIA Performance Primitives (NPP) is a library of functions for performing CUDA-accelerated 2D image and signal processing. This library is widely applicable for developers in these areas and is written to maximize flexibility while maintaining high performance.

nvJPEG Library

The nvJPEG Library provides high-performance, GPU-accelerated JPEG encoding and decoding functionality. This library is intended for image formats commonly used in deep learning and hyperscale multimedia applications.

nvJPEG2000 Library

The nvJPEG2000 library provides high-performance, GPU-accelerated JPEG2000 decoding functionality. This library is intended for JPEG2000 formatted images commonly used in deep learning, medical imaging, remote sensing, and digital cinema applications.

Archives

Find archived online documentation for CUDA Toolkit.

JetPack

The JetPack SDK is the most comprehensive solution for building AI applications. The JetPack installer can be used to flash the Jetson Developer Kit with the latest OS image and to install developer tools, libraries and APIs, samples, and documentation.

Jetson Linux

NVIDIA Jetson Linux supports development on the Jetson platform.

L4T APIs

The L4T APIs provide additional functionality to support application development. The APIs enable flexibility by providing better control over the underlying hardware blocks.

Archives

This archives section provides access to previously released JetPack, L4T, and L4T Multimedia documentation versions.

NVIDIA DeepStream SDK

The NVIDIA DeepStream SDK delivers a complete streaming analytics toolkit for situational awareness through computer vision, intelligent video analytics (IVA), and multi-sensor processing.

Overview

A platform to accelerate AI, HPC and Visualization GPU workflows and thus accelerate time to solution.

Catalog

The NGC Catalog is a curated set of GPU-optimized software. It consists of containers, pre-trained models, Helm charts for Kubernetes deployments and industry-specific AI toolkits with software development kits (SDKs). The content provided by NVIDIA and third-party ISVs simplify the building, customizing and integration of GPU-optimized software into workflows, accelerating the time to solutions for users.

Deploy Assets from NGC

NVIDIA tests NGC containers running AI, ML and DL workloads on NVIDIA GPUs on leading public clouds and on-prem servers through its NVIDIA certification programs. NVIDIA certified data center and edge servers, together with public cloud platforms, enable easy deployment of any NGC asset, in environments certified for performance and scalability by NVIDIA.

Private Registry

The NGC private registry provides you with a secure space to store and share custom containers, models, resources and helm charts within your enterprise. Take advantage of the deployment patterns you love from the Catalog -- but with your bespoke assets.

Platform

NVIDIA Omniverse Platform

NVIDIA Omniverse is a powerful, multi-GPU, real-time simulation and collaboration platform for 3D production pipelines based on Pixar’s Universal Scene Description and NVIDIA RTX.

NVIDIA Omniverse Launcher

The Omniverse Launcher is the first step into the Omniverse, giving immediate access to all the apps, connectors, and other downloads.

NVIDIA Omniverse Nucleus

At the core of Omniverse is a set of fundamental services known as Omniverse Nucleus that allow a variety of client applications, including digital content creation (DCC) tools, renderers, and microservices to share and modify authoritative representations of virtual worlds.

NVIDIA Omniverse RTX Renderer

NVIDIA Omniverse RTX Renderer is NVIDIA’s premiere real-time ray-tracing renderer for Omniverse. It exploits the RT Cores present on NVIDIA Omniverse RTX Renderer Capable GPUs for industry-leading quality at near-real-time speeds.

NVIDIA Omniverse Kit

NVIDIA Omniverse Kit is a toolkit for building native Omniverse applications and microservices. It’s built on a base framework known as Carbonite that provides a wide variety of functionality through a set of lightweight plug-ins.

NVIDIA Omniverse Kit API

Omniverse Kit is an editor and viewer for Omniverse. It’s also an SDK that can be used to develop applications that work with Omniverse.

NVIDIA Omniverse Deployment Server

This deployment guide is an effort to drive a deployment that is validated through a multi-vendor cooperation between NVIDIA and its system and ISV partners.

Apps

NVIDIA Omniverse Create

NVIDIA Omniverse Create is an Omniverse app for world building that allows users to assemble, light, simulate, and render large-scale scenes.

NVIDIA Omniverse View

NVIDIA Omniverse View is an Omniverse app that offers a simple yet powerful toolkit designed to visualize architectural and engineering projects with stunning, physically accurate rendering output.

NVIDIA Omniverse Audio2Face

NVIDIA Omniverse Audio2Face is a combination of AI-based technologies that generate facial motion and lip sync derived entirely from an audio source.

NVIDIA Omniverse Kaolin

NVIDIA Omniverse Kaolin is a powerful visualization tool that simplifies and accelerates 3D deep learning research using NVIDIA’s Kaolin PyTorch library.

NVIDIA Omniverse Machinima

NVIDIA Omniverse Machinima is an Omniverse app optimized for animated story telling.

Extending Omniverse

NVIDIA Omniverse Connectors

NVIDIA has built extensions and additional software layers on top of the open-source USD distribution that allow DCC tools and compute services to communicate easily with each other through the Omniverse Nucleus DB.

NVIDIA Omniverse Utilities

This includes file management and other utilities for Omniverse.

NVIDIA Omniverse Extensions

Omniverse Kit Extensions are plug-ins to Omniverse Kit that extend its capabilities—typically for the user interface or workflows. Extensions are offered with complete source code to help developers easily create, add, and modify the tools and workflows they need to be productive.

NVIDIA Omniverse Nucleus Services

Nucleus Services provide additional functionality that enriches the NVIDIA Omniverse user experience and works on a global Omniverse scale.

NVIDIA Iray

NVIDIA Iray rendering technology represents a comprehensive approach to state-of-the-art rendering for design visualization.

NVIDIA Iray Server

NVIDIA Iray Server is a network-attached rendering solution for Iray-compatible applications.

NVIDIA MDL

NVIDIA Material Definition Language (MDL) is a domain-specific language that describes the appearance of scene elements for a rendering process.

NVIDIA IndeX

NVIDIA IndeX is a 3D volumetric, interactive visualization SDK used by scientists and researchers to visualize and interact with massive datasets.

NVIDIA OptiX

The NVIDIA OptiX ray-tracing engine is a programmable system designed for NVIDIA GPUs and other highly parallel architectures.

NVIDIA Optical Flow SDK

The NVIDIA Optical Flow SDK provides a comprehensive set of APIs, samples, and documentation on Windows and Linux platforms for fully hardware-accelerated optical flow, which can be used for computing the relative motion of pixels between images.

NVIDIA Video Codec SDK

The NVIDIA Video Codec SDK provides a comprehensive set of APIs, samples, and documentation for fully hardware-accelerated video encoding, decoding, and transcoding on Windows and Linux platforms.