Built-in Operators and Extensions
The units of work of Holoscan applications are implemented within Operators, as described in the core concepts of the SDK. The operators included in the SDK provide domain-agnostic functionalities such as IO, machine learning inference, processing, and visualization, optimized for AI streaming pipelines, relying on a set of Core Technologies.
The operators below are defined under the holoscan::ops namespace for C++ and CMake, and under the holoscan.operators module in Python.
If you are new to Holoscan, a common starting pipeline is:
VideoStreamReplayerOp/V4L2VideoCaptureOp -> FormatConverterOp -> InferenceOp -> SegmentationPostprocessorOp -> HolovizOp.
Imaging and visualization
Class | Typical use | API reference |
|---|---|---|
| BayerDemosaicOp | Convert Bayer RAW camera input to RGB/BGR style images for downstream processing. | C++/Python |
| FormatConverterOp | Convert between tensor/image formats, memory layouts, and data types between operators. See Bring Your Own Model (BYOM). | C++/Python |
| HolovizOp | Visualize tensors, overlays, geometry, and results for debugging or runtime display. See Visualization. | C++/Python |
Inference and post-processing
Class | Typical use | API reference |
|---|---|---|
| InferenceOp | Run one or more AI models (including dependency-ordered pipelines) in streaming applications. See inference. | C++/Python |
| InferenceProcessorOp | Apply model-specific preprocessing/postprocessing around inference outputs. | C++/Python |
| SegmentationPostprocessorOp | Convert raw segmentation model outputs into masks/labels suitable for display or downstream logic. | C++/Python |
Source, replay, and recording
Class | Typical use | API reference |
|---|---|---|
| V4L2VideoCaptureOp | Capture live video frames from V4L2 devices on Linux systems. | C++/Python |
| VideoStreamReplayerOp | Replay recorded streams for deterministic development, benchmarking, and regression testing. See Video Replayer. | C++/Python |
| VideoStreamRecorderOp | Record tensor streams to disk for offline replay and reproducible experiments. | C++/Python |
Basic networking
Class | Typical use | API reference |
|---|---|---|
| PingTxOp | Emit synthetic scalar data to test graph connectivity and scheduling behavior. See native operator ping example and native operator ping example. | C++/Python |
| PingRxOp | Receive and inspect synthetic scalar data in tutorial or debugging pipelines. See native operator ping example and native operator ping example. | C++/Python |
| PingTensorTxOp | Emit synthetic tensor payloads to validate tensor paths and operator interoperability. | C++/Python |
| PingTensorRxOp | Receive synthetic tensor payloads for debugging C++/Python tensor flow behavior. | C++/Python |
For complete, end-to-end application examples that combine these operators, see the Holoscan SDK examples and the Holohub repository.
Given an instance of an operator class, you can print a human-readable description of its specification to inspect the inputs, outputs, and parameters that can be configured on that operator class:
std::cout << operator_object->spec()->description() << std::endl;
print(operator_object.spec)
The Holoscan SDK uses meta-programming with templating and std::any to support arbitrary data types. Because of this, some type information (and therefore values) might not be retrievable by the description API. If more details are needed, we recommend inspecting the list of Parameter members in the operator header to identify their type.
Holoscan UCX GXF Extension
The ucx_holoscan extension includes nvidia::holoscan::UcxHoloscanComponentSerializer which is a nvidia::gxf::ComponentSerializer that handles serialization of holoscan::Message and holoscan::Tensor types for transmission using the Unified Communication X (UCX) library. UCX is the library used by Holoscan SDK to enable communication of data between fragments in distributed applications. For more details see the Creating a Distributed Application page.
The UcxHoloscanComponentSerializer is intended for use in combination with other UCX components defined in the GXF UCX extension. Specifically, it can be used by the UcxEntitySerializer where it can operate alongside the UcxComponentSerializer that serializes GXF-specific types (nvidia::gxf::Tensor, nvidia::gxf::VideoBuffer, etc.). This way both GXF and Holoscan types can be serialized by distributed applications.
Holohub
Holohub is an open collection of reusable Holoscan operators, applications, workflows, and supporting tools designed to accelerate Holoscan-based development. You can browse the full set of operators directly in the Holohub repository or explore them with detailed documentation and search features on the Holohub documentation website.
For a quick overview of Holohub, see the Holohub Overview.