NVIDIA Holoscan is the AI sensor processing platform that combines hardware systems for low-latency sensor and network connectivity, optimized libraries for data processing and AI, and core microservices to run streaming, imaging, and other applications, from embedded to edge to cloud. It can be used to build streaming AI pipelines for a variety of domains, including Medical Devices, High Performance Computing at the Edge, Industrial Inspection and more.
In previous releases, the prefix Clara was used to define Holoscan as a platform designed initially for medical devices. As Holoscan has grown, its potential to serve other areas has become apparent. With version 0.4.0, we’re proud to announce that the Holoscan SDK is now officially built to be domain-agnostic and can be used to build sensor AI applications in multiple domains. Note that some of the content of the SDK (sample applications) or the documentation might still appear to be healthcare-specific pending additional updates. Going forward, domain specific content will be hosted on the HoloHub repository.
The Holoscan SDK assists developers by providing:
Various installation strategies
From containers, to python wheels, to source, from development to deployment environments, the Holoscan SDK comes in many packaging flavors to adapt to different needs. Find more information in the sdk installation section.
C++ and Python APIs
These APIs are now the recommended interface for the creation of application pipelines in the Holoscan SDK. See the Using the SDK section to learn how to leverage those APIs, or the Doxygen pages (C++/Python) for specific API documentation.
Built-in Operators and Extensions
The units of work of the Holoscan SDK are implemented within Operators, as described in the core concepts of the SDK. The operators included in the SDK provide domain-agnostic functionalities such as IO, machine learning inference, processing, and visualization, optimized for AI streaming pipelines, relying on a set of Core Technologies. Notably, NVIDIA partnered with AJA and Emergent Vision Technologies to provide support for their respective sensor IO, including operators in the Holoscan SDK that interface with their own respective SDKs. This guide provide more information on the existing extensions wrapped by Holoscan operators.
Minimal Examples
The Holoscan SDK provides a list of examples to illustrate specific capabilities of the SDK. Their source code can be found in the GitHub repository. The Holoscan by Example section provides step-by-step analysis of some of these examples to illustrate the innerworkings of the Holoscan SDK.
Reference Applications
This SDK includes multiple multiple sample applications to show how users can implement their own end-to-end inference pipeline for streaming use cases, as well as “bring your own model” (BYOM) capabilities.
Video Pipeline Latency Tool
To help developers make sense of the overall end-to-end latency that could be added to a video stream by augmenting it through a GPU-powered Holoscan platform such as the NVIDIA IGX Orin Developer Kit, the Holoscan SDK includes a Video Pipeline Latency Measurement Tool. This tool can be used to measure and estimate the total end-to-end latency of a video streaming application including the video capture, processing, and output using various hardware and software components that are supported by the Holoscan Developer Kits. The measurements taken by this tool can then be displayed with a comprehensive and easy-to-read visualization of the data.
Documentation
The Holoscan SDK documentation is composed of:
Build and run instructions specific to each installation strategy
Release notes on Github