Application Development#

Container Requirements#

Holoscan for Media is a platform for containerized software deployment. Requirements for developer containers are listed below.

Host OS Requirements#

Because Holoscan for Media applications are containerized, the host OS for development is not critical. However, to test features such as Single Root I/O Virtualization (SR-IOV) networking, a supported Linux distribution is recommended. Refer to Linux distributions officially supported by NVIDIA.

Base Container Requirements#

The base container is left to the developer. Our reference applications are based on Ubuntu 22.04 and 24.04.

Built on top of DeepStream SDK, Media Gateway has been designed to provide developers access to ST 2110 buffers and NMOS dynamic connectivity with the lowest possible friction. Media Gateway can be used as base container for development.

Deployment#

Holoscan for Media containerized applications are deployed using Helm, which allows them to request that appropriate resources be allocated on the cluster for the application, and to access network and graphics capabilities through operators. Developers can use the Media Gateway reference Helm deployment files as a reference to build their own, even if they are not using Media Gateway as a base container.

The example below is a typical deployment for an application based on Media Gateway. It pulls the appropriate image from NGC using the developer’s secret, which stores the NGC key. It requests isolated CPU cores, huge pages, and system memory. It also requests an interface on the high speed network.

image:
    repository: nvcr.io/nvidia/holoscan-for-media/nvds-nmos-bin
imagePullSecrets:
    - name: my-ngc-secret
pipelines:
    pipeline1:
        description: 1080p60-sender
        highSpeedNetwork:
            name: media-a-tx-net
        name: converter
        resources:
            limits:
                cpu: 4
                hugepages-2Mi: 4Gi
                memory: 8Gi
                openshift.io/media_a_tx_pool: 1
            requests:
                cpu: 4
                hugepages-2Mi: 4Gi
                memory: 8Gi
                openshift.io/media_a_tx_pool: 1

Key Software Technologies#

This section lists how developers can access key technologies using NVIDIA platform.

GPU Encoding and Decoding#

NVIDIA GPUs allow you to encode and decode compressed video from and to GPU memory with dedicated NVENC and NVDEC processing cores. Starting with GPU Blackwell generation, NVENC and NVDEC support YCbCr-4:2:2 10-bit encoding and decoding. GPU Support Matrix allows to check video capabilities encoding and decoding for any NVIDIA GPU.

While developers can access hardware acceleration for compressed streams using DeepStream SDK and appropriate pipeline with very little coding effort as done in Media Gateway, they can also access NVENC and NVDEC through other ways depending on their software architecture as follows:

NMOS#

Holoscan for Media developers can design NMOS compatible applications by using DeepStream and the source code for Media Gateway as inspiration. Developers also have access to dedicated NMOS containerized tools (NMOS registry and controller) for testing and validating their software.

It’s also possible to design your own NMOS component using code samples and utilities available here:

  • NVIDIA/nvnmos open-source NMOS Node implementation, which has a simple API, for common use cases

  • Sony nmos-cpp open-source implementation, which is more complex but flexible enough for a broader set of use cases

Other resources for getting started with NMOS are provided by the Advanced Media Workflow Association (AMWA) including:

PTP#

The Holoscan for Media platform provides PTP as a service. For development, PTP4L can also be run on a Linux host system with NVIDIA ConnectX-6 Dx and ConnectX-7 NICs. The NVIDIA Rivermax PTP Best Practice Guidelines outlines how a developer working on a bare metal Linux system can set up PTP for testing.

ST 2110#

Media Gateway#

Access to ST 2110 live streaming is made easy through Media Gateway reference application built on top of DeepStream SDK. Further details on SDP, relevant DeepStream pipelines and example of full deployment for Media Gateway are available to guide your development.

Rivermax#

It’s also possible to leverage Rivermax in native C++ code. After they are registered for access, a Rivermax developer can access functional code samples that are ideal reproducer vehicles for bug support:

  • media_sender (ST 2110 – Sender)

  • media_receiver (ST 2110 – Receiver)

  • media_node (NMOS wrapper for the above applications with a complete NMOS Media Node implementation that can send and receive)

Other open-source Rivermax samples are available in the Rivermax Dev Kit, a high-level C++ software kit designed to accelerate and simplify Rivermax application development.

Rivermax is a licensed technology. You can access a free development license by registering as a Rivermax developer. This license will be valid for three months but can be renewed on demand for as many times as needed to develop and support your product.

Access Cluster with VS Code#

The Kubernetes cluster can be accessed and managed using Visual Studio (VS) Code.

Install kubectl and then the Kubernetes and Dev Containers extensions in VS Code. These will let you manage the cluster and work with containers directly from the editor.

Connecting#

The cluster can be accessed remotely, that is using a different machine than where the cluster was created from, or from the same machine.

Connect to Cluster Remotely#

  1. Locate the kubeconfig file on the machine where the cluster was created from. This file contains the configuration, and credentials needed to access the cluster.

    The configuration can be found by running the following command:

    cat ~/.kube/config

  2. Copy the file to the appropriate location on your machine.

    Refer to Organizing Cluster Access Using kubeconfig Files for more information.

  3. Open VS Code on the machine and navigate to the Kubernetes extension.

    picture of the Kubernetes extension in Visual Code

  4. Go to Set Kubeconfig option.

    A window will pop up.

  5. Select Add new kubeconfig.

    picture of adding a new kubeconfig in Visual Code

  6. Choose the kubeconfig file.

Connect Locally#

If the same machine is being used that is hosting the cluster, VS Code will automatically pick up the kubeconfig file.

Usage#

After it is connected, the cluster can be viewed in the Clusters section of the Kubernetes extension. From here, all running pods can be viewed.

List all running pods:

picture showing all the running pods in Visual Code

Attach Visual Code by right-clicking the target container:

picture of attaching Visual Code

Edit, or modify files with Visual Code:

picture of browsing, editing, modify files in Visual Code

Testing#

Developers can use the Media Gateway container to help test their application, for example, to transmit or receive:

Changing the DeepStream pipeline in the Helm chart values allows developers to create different instances of Media Gateway with different behaviors (for example, ST 2110 playout from file, ST 2110 ingest to file or live ST 2110 to SRT streaming).

Using Chrome Remote Desktop, developers can dynamically connect Media Gateway to ST 2110 sources or receivers using the NMOS controller UI, and you can also run VLC Media Player to visualize a stream. For debugging purposes, ST 2110 streams can be encoded and transmitted, for example using SRT for viewing in VLC as shown in this reference pipeline.

Profiling#

We recommend that developers profile their applications with Nsight Developer tools to identify bottlenecks and optimize performance. This is key to provide highest possible density to end users when running multiple containerized software on the cluster.

Depending on your application, different flavors of Nsight tools are available:

  • Nsight Systems to identify system level behavior and possible bottlenecks, supporting both CPU and GPU calls, including NVENC and NVDEC GPU units for compressed video pipelines

  • Nsight Graphics to profile graphical applications

  • Nsight Compute for CUDA code profiling

You can also integrate markers using NVIDIA Tools Extension SDK (NVTX) to annotate events, code ranges, and resources in your applications.

You can also profile applications while running on Holoscan for Media using NVIDIA Nsight Cloud, allowing you to profile and capture data from the CPU, GPU, network, storage, workload, API, and more in a Kubernetes environment.

Packaging#

After the application is developed, validated and optimized, a Helm chart can be created to deploy the application, incorporating appropriate default resource requests. Developers can use Kubernetes Dashboard to look at CPU and memory use and Platform Monitoring to check GPU and networking effective use while the application is running under appropriate load testing.