Introduction to NVIDIA JetPack SDK
NVIDIA JetPack SDK is the most comprehensive solution for building AI applications. JetPack SDK provides a full development environment for hardware-accelerated AI-at-the-edge development. JetPack SDK includes Jetson Linux Driver Package with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. It also includes samples, documentation, and developer tools for both host computer and developer kit, and supports higher level SDKs such as DeepStream for streaming video analytics, Isaac for robotics and Riva for conversational AI.
JetPack 6.0 components include the following:
OS Image
JetPack 6.0 includes NVIDIA Jetson Linux 36.3 which includes the Linux Kernel 5.15, UEFI based bootloader, Ubuntu 22.04 based root file system, NVIDIA drivers, necessary firmwares, toolchain and more. See the release notes for all the highlights in Jetson Linux 36.3.
Libraries
TensorRT and cuDNN for high-performance deep learning applications.
TensorRT is a high-performance deep learning inference runtime for image classification, segmentation, and object detection neural networks. It speeds up deep learning inference as well as reducing the runtime memory footprint for convolutional and deconv neural networks.
CUDA Deep Neural Network library provides high-performance primitives for deep learning frameworks. It includes support for convolutions, activation functions, and tensor transformations.
CUDA for GPU accelerated applications across multiple domains.
NVIDIA Container Runtime for containerized GPU accelerated applications.
The container runtime enables creation, distribution, and use of containerized GPU accelerated applications.
-
The Jetson Multimedia API package provides low level APIs for flexible application development.
Camera application API: libargus offers a low-level frame-synchronous API for camera applications, with per frame camera parameter control, multiple (including synchronized) camera support, and EGL stream outputs. RAW output CSI cameras needing ISP can be used with either libargus or GStreamer plugin. In either case, the V4L2 media-controller sensor driver API is used.
Sensor driver API: V4L2 framework enables video decode, encode, format conversion and scaling functionality. V4L2 for encode opens up many features like bit rate control, quality presets, low latency encode, temporal tradeoff, motion vector maps, and more.
VPI and OpenCV, and for visual computing applications.
VPI (Vision Programming Interface), a software library that provides Computer Vision / Image Processing algorithms implemented on PVA (Programmable Vision Accelerator), GPU and CPU.
OpenCV is the leading open source library for computer vision, image processing and machine learning, and now features GPU acceleration for real-time operation.
Samples
There are a number of samples demonstrating use of JetPack components. These are included in the reference filesystem, and can be compiled on the developer kit.
JetPack Component |
Sample locations on reference filesystem |
|---|---|
TensorRT |
|
cuDNN |
|
CUDA |
|
MM API |
|
OpenCV |
|
VPI |
|
Developer Tools
JetPack includes developer tools for application development, debugging, profiling, and optimization. Some tools are used directly on a Jetson system, and others run on a Linux host computer connected to a Jetson system.
Tools for application development and debugging:
- NVIDIA Nsight Eclipse Edition for development of GPU accelerated applications.
NVIDIA Nsight Eclipse Edition (bundled with CUDA Toolkit) is a full-featured IDE powered by the Eclipse platform. It provides an all-in-one integrated environment to edit, cross-compile, and debug CUDA-C applications. Nsight Eclipse Edition supports a rich set of commercial and free plugins. Runs on Linux host computer. Supports all Jetson products.
- CUDA-GDB for application debugging.
CUDA-GDB is a command line tool bundled with CUDA Toolkit that delivers a seamless debugging experience allowing you to debug both the CPU and GPU portions of your application simultaneously. Runs on Jetson system or on Linux host computer. Supports all Jetson products.
- CUDA-MEMCHECK for debugging application memory errors.
CUDA-MEMCHECK is a command line tool bundled with CUDA Toolkit that detects the source and cause of memory access errors in your GPU code. CUDA-MEMCHECK also reports runtime execution errors, identifying situations that could otherwise result in an “unspecified launch failure” error when your application is running. Runs on Jetson system. Supports all Jetson products.
Tools for application profiling and optimization:
- NVIDIA Nsight Systems for application profiling across GPU and CPU.
NVIDIA Nsight Systems is a low overhead system-wide profiling tool, providing the insights developers need to analyze and optimize software performance. It uses GPU tracing, CPU sampling and tracing, and OS thread state tracing to visualize an application’s algorithms, helping developers identify the largest opportunities to optimize their code. Runs on Linux host computer. Supports all Jetson products.
- nvprof for application profiling across GPU and CPU.
nvprof is a command line tool bundled with CUDA Toolkit that enables you to collect and view profiling data, i.e., a timeline of CUDA-related activities on both CPU and GPU. Runs on Jetson system. Supports all Jetson products.
- Visual Profiler for application profiling across GPU and CPU.
Visual Profiler (bundled with CUDA Toolkit) will be deprecated in the future. Developers should use NVIDIA Nsight Systems for application sampling and tracing across GPU and CPU, and Nsight Compute for CUDA kernel profiling. Runs on Linux host computer. Supports all Jetson products.
- NVIDIA Nsight Graphics for graphics application debugging and profiling.
NVIDIA Nsight Graphics is a standalone application for debugging and profiling graphics applications. Powerful analysis tools help identify optimization opportunities. Runs on Linux host computer. Supports all Jetson products.
- NVIDIA Nsight Compute for interactive CUDA kernel profiling.
NVIDIA Nsight Compute (bundled with CUDA Toolkit) is an interactive kernel profiler for CUDA applications. It provides detailed performance metrics for analysis and enables results comparison between baselines and the current run. Nsight Compute can be extended with analysis scripts for post-processing results. Runs on Linux host computer. Supports Jetson AGX Xavier and Jetson Xavier NX.
- NVIDIA Nsight Deep Learning Designer for developing neural networks.
NVIDIA Nsight Deep Learning Designer is an integrated development environment that helps developers efficiently design and develop deep neural networks for in-app inference.
- NVIDIA Nsight Compute CLI for CUDA kernel profiling.
NVIDIA Nsight Compute CLI (bundled with CUDA Toolkit) provides a non-interactive way to profile applications from the command line. It can print the results directly on the command line or store them in a report file. It can also be used to simply launch the target application and then suspend it until another tool attaches for profiling. Runs on Jetson system or on Linux host computer. Supports Jetson AGX Xavier and Jetson Xavier NX.