SDK Installation
The section below refers to the installation of the Holoscan SDK referred to as the development stack, designed for the NVIDIA IGX Developer Kits, and for x86_64 Linux compute platforms, ideal for development and testing of the SDK.
For Holoscan Developer Kits such as the IGX Orin Developer Kit, an alternative option is the deployment stack, based on OpenEmbedded (Yocto build system) instead of Ubuntu. This is recommended to limit your stack to the software components strictly required to run your Holoscan application. The runtime Board Support Package (BSP) can be optimized with respect to memory usage, speed, security and power requirements.
Set up your developer kit:
Developer Kit | User Guide | L4T | GPU Mode |
|---|---|---|---|
| NVIDIA IGX Orin | Guide | IGX OS 1.2.0-lws2.5.0 LWS2 | iGPU or dGPU |
You’ll need the following to use the Holoscan SDK on x86_64:
OS: Ubuntu 22.04 (GLIBC >= 2.35)
NVIDIA discrete GPU (dGPU)
Ampere or above recommended for best performance
Quadro/NVIDIA RTX necessary for GPUDirect RDMA support
Tested with NVIDIA Quadro RTX 6000 and NVIDIA RTX A6000
NVIDIA dGPU drivers: 535 or above
For RDMA Support (RoCE over ConnectX and/or GPUDirect RDMA), follow the instructions in the Enabling RDMA section.
Additional software dependencies might be needed based on how you choose to install the SDK (see section below).
Refer to the Additional Setup and Third-Party Hardware Setup sections for additional prerequisites.
Authentication
To access LWS2 installations, you will first need an API Key from NGC. Select NGC Catalog as one of the services. Your API key should begin with nvapi-.
If you plan to only use the Docker container, skip to the installation section. If you plan to install the Debian package or Python wheel, you will first need to configure paywall authentication tools in step 1 below.
buildauthcliandauthxsentrypaywall tools already come installed on IGX OS LWS2 (iGPU and dGPU). For other platforms, visit the NGC NVIDIA Paywall Utilities for LTS portal and manually download the correspondingauthxsentryandbuildauthclipackages from the File Browser. Then, complete the local setup:sudo mv path/to/authxsentry /usr/bin/authxsentry sudo tar -zx path/to/buildauthcli-linux-amd64-v2.0.tar.gz -C "/usr/bin/" sudo chmod +x /usr/bin/authxsentry sudo chmod +x /usr/bin/buildauth
On both IGX and x86_64, run
authxsentrywith the NGC CLI API key you generated above:authxsentry <nvapi-xxx> # Successful setup will end with `Login complete.`
Installation
We provide multiple ways to install and run the Holoscan SDK:
Login to the NVIDIA container registry:
docker login nvcr.io
# Username: $oauthtoken
# Password: <NGC API key>
You can then pull the image from the NGC LWS2 repository:
dGPU (x86_64)
docker pull nvcr.io/nvidia/holoscan-ltsb2:23.10.08-lws2.6.1-dgpu
dGPU (IGX Orin dGPU)
docker pull nvcr.io/nvidia/holoscan-ltsb2:23.10.08-lws2.5.1-dgpu
iGPU (IGX Orin iGPU)
docker pull nvcr.io/nvidia/holoscan-ltsb2:23.10.08-lws2.5.1-igpu
To install the Holoscan LWS2 Debian packages, apt must point to the LWS2 repositories. That is already configured on IGX OS LWS2 (iGPU and dGPU). On other platforms, run the following to configure APT remote package sources:
sudo tee /etc/apt/sources.list.d/lws2.list >/dev/null <<'EOF'
deb [arch=amd64 trusted=yes] https://disthub.nvidia.com/artifactory/oss-lws-deb-stable jammy main
deb [arch=amd64 trusted=yes] https://disthub.nvidia.com/artifactory/nvaie-lws2-deb-stable lws lws2
EOF
No repository signing key is available for disthub.nvidia.com. Use trusted=yes for the dedicated LWS2 repository entries.
Run the following to install the Holoscan SDK Debian package and its supported dependencies:
sudo apt update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/lws2.list
sudo apt install holoscan -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/lws2.list
This will not install dependencies needed for the Torch nor ONNXRuntime inference backends. To do so, install transitive dependencies by adding the --install-suggests flag to apt install holoscan, and refer to the support matrix below for links to install libtorch and onnxruntime.
To use the Python module included in the Debian package, include the path below in your Python path:
export PYTHONPATH="/opt/nvidia/holoscan/python/lib"
Run the following to install the Holoscan SDK Python wheel package. The LWS2 package repository is already configured on IGX OS LWS2. For other platforms,
add --index-url and --extra-index-url to install from supported locations.
python -m pip install \
--index-url https://disthub.nvidia.com/artifactory/api/pypi/nvaie-lws2-pypi-stable/simple \
--extra-index-url https://disthub.nvidia.com/artifactory/api/pypi/oss-lws2-pypi-stable/simple \
holoscan~=1.0.3
For x86_64, ensure that the CUDA Toolkit is pre-installed.
Not sure what to choose?
The Holoscan container image on NGC is the safest way to ensure all the dependencies are present with the expected versions (including torch 8). It is the simplest way to run the embedded examples, while still allowing you to create your own C++ and Python Holoscan application on top of it. These benefits come with the standard inconvenience that exist when using Docker, such as more complex run instructions for proper configuration. Also, supporting Rivermax or the CLI require more work than the other solutions at this time.
If you are confident in your ability to manage dependencies on your own in your host environment, the Holoscan Debian package should provide all the capabilities needed to use the Holoscan SDK.
If you are not interested in the C++ API but just need to work in Python, you can use the Holoscan Python wheels. While they are the easiest solution to get started, you might need additional work to setup your environment with adequate dependencies depending on your needs.
Docker Container | Debian Package | Python Wheels | |
|---|---|---|---|
| Runtime libraries | Included | Included | Included |
| Python module | 3.10 | 3.10 | 3.10 |
| C++ headers and CMake config |
Included | Included | N/A |
| Examples (+ source) | Included | Included | retrieve from GitHub |
| Sample datasets | Included | retrieve from Container | retrieve from Container |
| CUDA runtime 1 | Included | automatically 2 installed |
require manual installation |
| NPP support 3 | Included | automatically 2 installed |
require manual installation |
| TensorRT support 4 | Included | automatically 2 installed |
require manual installation |
| Vulkan support 5 | Included | automatically 2 installed |
require manual installation |
| V4L2 support 6 | Included | automatically 2 installed |
require manual installation |
| Torch support 7 | Included | require manual 8 installation |
require manual 8 installation |
| ONNX Runtime support 9 | Included | require manual 10 installation |
require manual 10 installation |
| MOFED RoCE support 11 | Included | require manual 13 installation |
require manual 13 installation |
| Rivermax support 11 | add on top 12 of the image |
require manual 13 installation |
require manual 13 installation |
Need more control over the SDK?
The Holoscan SDK source repository is open-source and provides reference implementations as well as infrastructure for building the SDK yourself.
We only recommend building the SDK from source if you need to build it with debug symbols or other options not used as part of the published packages. If you want to write your own operator or application, we recommend you use the pre-built SDK as a dependency. External modifications to the SDK are not supported under the Holoscan SDK Long-term Whole Stack program.
CUDA 12 is required. Already installed on NVIDIA IGX developer kits.
Debian installation on x86_64 requires disthub.nvidia.com package source configuration to automatically install supported dependencies.
NPP 12 needed for the FormatConverter and BayerDemosaic operators. Already installed on NVIDIA IGX developer kits.
TensorRT 8.6.2 and cuDNN 8.9 needed for the Inference operator. Already installed on NVIDIA IGX developer kits.
Vulkan 1.3.204+ loader needed for the HoloViz operator (+ libegl1 for headless rendering). Already installed on NVIDIA IGX developer kits.
V4L2 1.22+ needed for the V4L2 operator. Already installed on NVIDIA IGX developer kits.
Torch support requires LibTorch 2.1+, TorchVision 0.16+, OpenBLAS 0.3.20+, OpenMPI (aarch64 only), MKL 2021.1.1 (x86_64 only), libpng and libjpeg.
To install LibTorch and TorchVision, extract them from the DLFW PyTorch Long-Term Support Branch container. LTSB customers should inquire if needed.
ONNX Runtime 1.15.1+ needed for the Inference operator. Note that ONNX models are also supported through the TensorRT backend of the Inference Operator.
To install ONNX Runtime, extract it from the Holoscan Long-Term Support Branch container (in /opt/onnxruntime). LTSB customers should inquire if needed.
Tested with MOFED 23.10 and Rivermax SDK 1.40.11
Rivermax SDK and OFED drivers and libraries are installed on NVIDIA developer kits with SDKM though the Rivermax SDK program
Ensure to install the Rivermax license file on the host and mount it in the container