holoscan.operators

This module provides a Python API to underlying C++ API Operators.

class holoscan.operators.AJASourceOp

Bases: holoscan.core._core.Operator

Operator to get a video stream from an AJA capture card.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.aja_source._aja_source.AJASourceOp, fragment: holoscan.core._core.Fragment, device: str = '0', channel: holoscan.operators.aja_source._aja_source.NTV2Channel = <NTV2Channel.NTV2_CHANNEL1: 0>, width: int = 1920, height: int = 1080, framerate: int = 60, rdma: bool = False, enable_overlay: bool = False, overlay_channel: holoscan.operators.aja_source._aja_source.NTV2Channel = <NTV2Channel.NTV2_CHANNEL2: 1>, overlay_rdma: bool = True, name: str = 'aja_source') → None

Operator to get a video stream from an AJA capture card.

Named inputs:

overlay_buffer_input: nvidia::gxf::VideoBuffer (optional)

The operator does not require a message on this input port in order for compute to be called. If a message is found, and enable_overlay is True, the image will be mixed with the image captured by the AJA card. If enable_overlay is False, any message on this port will be ignored.

Named outputs:

video_buffer_output: nvidia::gxf::VideoBuffer

The output video frame from the AJA capture card. If overlay_rdma is True, this video buffer will be on the device, otherwise it will be in pinned host memory.

overlay_buffer_output: nvidia::gxf::VideoBuffer (optional)

This output port will only emit a video buffer when enable_overlay is True. If overlay_rdma is True, this video buffer will be on the device, otherwise it will be in pinned host memory.

Parameters

fragment

The fragment that the operator belongs to.

device

The device to target (e.g. “0” for device 0)

channel

The camera NTV2Channel to use for output.

width

Width of the video stream.

height

Height of the video stream.

framerate

Frame rate of the video stream.

rdma

Boolean indicating whether RDMA is enabled.

enable_overlay

Boolean indicating whether a separate overlay channel is enabled.

overlay_channel

The camera NTV2Channel to use for overlay output.

overlay_rdma

Boolean indicating whether RDMA is enabled for the overlay.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.aja_source._aja_source.AJASourceOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.aja_source._aja_source.AJASourceOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.BayerDemosaicOp

Bases: holoscan.core._core.Operator

Bayer Demosaic operator.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp) -> None

Bayer Demosaic operator.

2. __init__(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp, fragment: holoscan.core._core.Fragment, pool: holoscan.resources._resources.Allocator, cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, in_tensor_name: str = ‘’, out_tensor_name: str = ‘’, interpolation_mode: int = 0, bayer_grid_pos: int = 2, generate_alpha: bool = False, alpha_value: int = 255, name: str = ‘format_converter’) -> None

Bayer Demosaic operator.

Named inputs:

receiver: nvidia::gxf::Tensor or nvidia::gxf::VideoBuffer

The input video frame to process. If the input is a VideoBuffer it must be an 8-bit unsigned grayscale video (nvidia::gxf::VideoFormat::GXF_VIDEO_FORMAT_GRAY). The video buffer may be in either host or device memory (a host->device copy is performed if needed). If a video buffer is not found, the input port message is searched for a tensor with the name specified by in_tensor_name. This must be a device tensor in either 8-bit or 16-bit unsigned integer format.

Named outputs:

transmitter: nvidia::gxf::Tensor

The output video frame after demosaicing. This will be a 3-channel RGB image if alpha_value is True, otherwise it will be a 4-channel RGBA image. The data type will be either 8-bit or 16-bit unsigned integer (matching the bit depth of the input). The name of the tensor that is output is controlled by out_tensor_name.

Parameters

fragment

The fragment that the operator belongs to.

pool

Memory pool allocator used by the operator.

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

in_tensor_name

The name of the input tensor.

out_tensor_name

The name of the output tensor.

interpolation_mode

The interpolation model to be used for demosaicing. Values available at: https://docs.nvidia.com/cuda/npp/group__typedefs__npp.html#ga2b58ebd329141d560aa4367f1708f191

bayer_grid_pos

The Bayer grid position (default of 2 = GBRG). Values available at: https://docs.nvidia.com/cuda/npp/group__typedefs__npp.html#ga5597309d6766fb2dffe155990d915ecb

generate_alpha

Generate alpha channel.

alpha_value

Alpha value to be generated if generate_alpha is set to True.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.FormatConverterOp

Bases: holoscan.core._core.Operator

Format conversion operator.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.format_converter._format_converter.FormatConverterOp, fragment: holoscan.core._core.Fragment, pool: holoscan.resources._resources.Allocator, out_dtype: str, in_dtype: str = '', in_tensor_name: str = '', out_tensor_name: str = '', scale_min: float = 0.0, scale_max: float = 1.0, alpha_value: int = 255, resize_height: int = 0, resize_width: int = 0, resize_mode: int = 0, out_channel_order: List[int] = [], cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'format_converter') → None

Format conversion operator.

Named inputs:

source_video: nvidia::gxf::Tensor or nvidia::gxf::VideoBuffer

The input video frame to process. If the input is a VideoBuffer it must be in format GXF_VIDEO_FORMAT_RGBA, GXF_VIDEO_FORMAT_RGB or GXF_VIDEO_FORMAT_NV12. This video buffer may be in either host or device memory (a host->device copy is performed if needed). If a video buffer is not found, the input port message is searched for a tensor with the name specified by in_tensor_name. This must be a device tensor in one of several supported formats (unsigned 8-bit int or float32 graycale, unsigned 8-bit int RGB or RGBA, YUV420 or NV12).

Named outputs:

tensor: nvidia::gxf::Tensor

The output video frame after processing. The shape, data type and number of channels of this output tensor will depend on the specific parameters that were set for this operator. The name of the Tensor transmitted on this port is determined by out_tensor_name.

Parameters

fragment

The fragment that the operator belongs to.

pool

Memory pool allocator used by the operator.

out_dtype

Destination data type (e.g. “rgb888” or “rgba8888”).

in_dtype

Source data type (e.g. “rgb888” or “rgba8888”).

in_tensor_name

The name of the input tensor (default is the empty string, “”).

out_tensor_name

The name of the output tensor (default is the empty string, “”).

scale_min

Output will be clipped to this minimum value.

scale_max

Output will be clipped to this maximum value.

alpha_value

Unsigned integer in range [0, 255], indicating the alpha channel value to use when converting from RGB to RGBA.

resize_height

Desired height for the (resized) output. Height will be unchanged if resize_height is 0.

resize_width

Desired width for the (resized) output. Width will be unchanged if resize_width is 0.

resize_mode

Resize mode enum value corresponding to NPP’s nppiInterpolationMode (default=NPPI_INTER_CUBIC).

channel_order

Sequence of integers describing how channel values are permuted.

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.format_converter._format_converter.FormatConverterOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.format_converter._format_converter.FormatConverterOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.HolovizOp(fragment, allocator=None, receivers=(), tensors=(), color_lut=(), window_title='Holoviz', display_name='DP-0', width=1920, height=1080, framerate=60, use_exclusive_display=False, fullscreen=False, headless=False, enable_render_buffer_input=False, enable_render_buffer_output=False, enable_camera_pose_output=False, font_path='', cuda_stream_pool=None, name='holoviz_op')

Bases: holoscan.operators.holoviz._holoviz.HolovizOp

Holoviz visualization operator using Holoviz module.

This is a Vulkan-based visualizer.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

InputSpec	InputSpec for the HolovizOp operator.
add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

DepthMapRenderMode
InputType
OperatorType

class DepthMapRenderMode

Bases: pybind11_builtins.pybind11_object

Members:

POINTS

LINES

TRIANGLES

Attributes

name

value

LINES = <DepthMapRenderMode.LINES: 1>

POINTS = <DepthMapRenderMode.POINTS: 0>

TRIANGLES = <DepthMapRenderMode.TRIANGLES: 2>

__init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.DepthMapRenderMode, value: int) → None

property name

property value

class InputSpec

Bases: pybind11_builtins.pybind11_object

InputSpec for the HolovizOp operator.

Parameters

tensor_name

The tensor name for this input.

type

The type of data that this tensor represents.

Attributes

type	(holoscan.operators.HolovizOp.InputType) The type of data that this tensor represents.
opacity	(float) The opacity of the object. Must be in range [0.0, 1.0] where 1.0 is fully opaque.
priority	(int) Layer priority, determines the render order. Layers with higher priority values are rendered on top of layers with lower priority.
color	(4-tuple of float) RGBA values in range [0.0, 1.0] for rendered geometry.
line_width	(float) Line width for geometry made of lines.
point_size	(float) Point size for geometry made of points.
text	(sequence of str) Sequence of strings used when type is HolovizOp.InputType.TEXT.
depth_map_render_mode	(holoscan.operators.HolovizOp.DepthMapRenderMode) The depth map render mode. Used only if type is HolovizOp.InputType.DEPTH_MAP or HolovizOp.InputType.DEPTH_MAP_COLOR.
views	(list of HolovizOp.InputSpec.View) Sequence of layer views. By default a layer will fill the whole window. When using a view, the layer can be placed freely within the window. When multiple views are specified, the layer is drawn multiple times using the specified layer views.

Methods

View	View for the InputSpec of a HolovizOp operator.
description(self)	Returns

class View

Bases: pybind11_builtins.pybind11_object

View for the InputSpec of a HolovizOp operator.

Notes

Layers can also be placed in 3D space by specifying a 3D transformation matrix. Note that for geometry layers there is a default matrix which allows coordinates in the range of [0 … 1] instead of the Vulkan [-1 … 1] range. When specifying a matrix for a geometry layer, this default matrix is overwritten.

When multiple views are specified, the layer is drawn multiple times using the specified layer views.

It’s possible to specify a negative term for height, which flips the image. When using a negative height, one should also adjust the y value to point to the lower left corner of the viewport instead of the upper left corner.

Attributes

offset_x, offset_y	(float) Offset of top-left corner of the view. (0, 0) is the upper left and (1, 1) is the lower right.
width	(float) Normalized width (range [0.0, 1.0]).
height	(float) Normalized height (range [0.0, 1.0]).
matrix	(sequence of float) 16-elements representing a 4x4 transformation matrix.

__init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec.View) → None

View for the InputSpec of a HolovizOp operator.

Notes

Layers can also be placed in 3D space by specifying a 3D transformation matrix. Note that for geometry layers there is a default matrix which allows coordinates in the range of [0 … 1] instead of the Vulkan [-1 … 1] range. When specifying a matrix for a geometry layer, this default matrix is overwritten.

When multiple views are specified, the layer is drawn multiple times using the specified layer views.

It’s possible to specify a negative term for height, which flips the image. When using a negative height, one should also adjust the y value to point to the lower left corner of the viewport instead of the upper left corner.

Attributes

offset_x, offset_y	(float) Offset of top-left corner of the view. (0, 0) is the upper left and (1, 1) is the lower right.
width	(float) Normalized width (range [0.0, 1.0]).
height	(float) Normalized height (range [0.0, 1.0]).
matrix	(sequence of float) 16-elements representing a 4x4 transformation matrix.

property height

property matrix

property offset_x

property offset_y

property width

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec, arg0: str, arg1: holoscan.operators.holoviz._holoviz.HolovizOp.InputType) -> None

InputSpec for the HolovizOp operator.

Parameters

tensor_name

The tensor name for this input.

type

The type of data that this tensor represents.

Attributes

type	(holoscan.operators.HolovizOp.InputType) The type of data that this tensor represents.
opacity	(float) The opacity of the object. Must be in range [0.0, 1.0] where 1.0 is fully opaque.
priority	(int) Layer priority, determines the render order. Layers with higher priority values are rendered on top of layers with lower priority.
color	(4-tuple of float) RGBA values in range [0.0, 1.0] for rendered geometry.
line_width	(float) Line width for geometry made of lines.
point_size	(float) Point size for geometry made of points.
text	(sequence of str) Sequence of strings used when type is HolovizOp.InputType.TEXT.
depth_map_render_mode	(holoscan.operators.HolovizOp.DepthMapRenderMode) The depth map render mode. Used only if type is HolovizOp.InputType.DEPTH_MAP or HolovizOp.InputType.DEPTH_MAP_COLOR.
views	(list of HolovizOp.InputSpec.View) Sequence of layer views. By default a layer will fill the whole window. When using a view, the layer can be placed freely within the window. When multiple views are specified, the layer is drawn multiple times using the specified layer views.
2. __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec, arg0: str, arg1: str) -> None

property color

property depth_map_render_mode

description(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec) → str

Returns

description

YAML string representation of the InputSpec class.

property line_width

property opacity

property point_size

property priority

property text

property type

property views

class InputType

Bases: pybind11_builtins.pybind11_object

Members:

UNKNOWN

COLOR

COLOR_LUT

POINTS

LINES

LINE_STRIP

TRIANGLES

CROSSES

RECTANGLES

OVALS

TEXT

DEPTH_MAP

DEPTH_MAP_COLOR

POINTS_3D

LINES_3D

LINE_STRIP_3D

TRIANGLES_3D

Attributes

name

value

COLOR = <InputType.COLOR: 1>

COLOR_LUT = <InputType.COLOR_LUT: 2>

CROSSES = <InputType.CROSSES: 7>

DEPTH_MAP = <InputType.DEPTH_MAP: 11>

DEPTH_MAP_COLOR = <InputType.DEPTH_MAP_COLOR: 12>

LINES = <InputType.LINES: 4>

LINES_3D = <InputType.LINES_3D: 14>

LINE_STRIP = <InputType.LINE_STRIP: 5>

LINE_STRIP_3D = <InputType.LINE_STRIP_3D: 15>

OVALS = <InputType.OVALS: 9>

POINTS = <InputType.POINTS: 3>

POINTS_3D = <InputType.POINTS_3D: 13>

RECTANGLES = <InputType.RECTANGLES: 8>

TEXT = <InputType.TEXT: 10>

TRIANGLES = <InputType.TRIANGLES: 6>

TRIANGLES_3D = <InputType.TRIANGLES_3D: 16>

UNKNOWN = <InputType.UNKNOWN: 0>

__init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputType, value: int) → None

property name

property value

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.holoviz._holoviz.HolovizOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, receivers: List[holoscan.core._core.IOSpec] = [], tensors: List[holoscan::ops::HolovizOp::InputSpec] = [], color_lut: List[List[float]] = [], window_title: str = 'Holoviz', display_name: str = 'DP-0', width: int = 1920, height: int = 1080, framerate: int = 60, use_exclusive_display: bool = False, fullscreen: bool = False, headless: bool = False, enable_render_buffer_input: bool = False, enable_render_buffer_output: bool = False, enable_camera_pose_output: bool = False, font_path: str = '', cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'holoviz_op') → None

Holoviz visualization operator using Holoviz module.

This is a Vulkan-based visualizer.

Named inputs:

receivers: multi-receiver accepting nvidia::gxf::Tensor and/or nvidia::gxf::VideoBuffer

Any number of upstream ports may be connected to this receivers port. This port can accept either VideoBuffers or Tensors. These inputs can be in either host or device memory. Each tensor or video buffer will result in a layer. The operator autodetects the layer type for certain input types (e.g. a video buffer will result in an image layer). For other input types or more complex use cases, input specifications can be provided either at initialization time as a parameter or dynamically at run time (via input_specs). On each call to compute, tensors corresponding to all names specified in the tensors parameter must be found or an exception will be raised. Any extra, named tensors not present in the tensors parameter specification (or optional, dynamic input_specs input) will be ignored.

input_specs: list[holoscan.operators.HolovizOp.InputSpec] (optional)

A list of InputSpec objects. This port can be used to dynamically update the overlay specification at run time. No inputs are required on this port in order for the operator to compute.

render_buffer_input: nvidia::gxf::VideoBuffer (optional)

An empty render buffer can optionally be provided. The video buffer must have format GXF_VIDEO_FORMAT_RGBA and be in device memory. This input port only exists if enable_render_buffer_input was set to True, in which case compute will only be called when a message arrives on this input.

Named outputs:

render_buffer_output: nvidia::gxf::VideoBuffer (optional)

Output for a filled render buffer. If an input render buffer is specified, it is using that one, else it allocates a new buffer. The video buffer will have format GXF_VIDEO_FORMAT_RGBA and will be in device memory. This output is useful for offline rendering or headless mode. This output port only exists if enable_render_buffer_output was set to True.

camera_pose_output: std::array<float, 16> (optional)

The camera pose. The parameters returned represent the values of a 4x4 row major projection matrix. This output port only exists if enable_camera_pose_output was set to True.

Parameters

fragment

The fragment that the operator belongs to.

allocator

Allocator used to allocate render buffer output. If None, will default to holoscan.core.UnboundedAllocator.

receivers

List of input receivers.

tensors

List of input tensors. Each tensor is defined by a dictionary where the ‘name’ key must correspond to a tensor sent to the operator’s input. See the notes section below for further details on how the tensor dictionary is defined.

color_lut

Color lookup table for tensors of type ‘color_lut’. Should be shape (n_colors, 4).

window_title

Title on window canvas.

display_name

In exclusive mode, name of display to use as shown with xrandr.

width

Window width or display resolution width if in exclusive or fullscreen mode.

height

Window height or display resolution width if in exclusive or fullscreen mode.

framerate

Display framerate in Hz if in exclusive mode.

use_exclusive_display

Enable exclusive display.

fullscreen

Enable fullscreen window.

headless

Enable headless mode. No window is opened, the render buffer is output to port render_buffer_output.

enable_render_buffer_input

If True, an additional input port, named ‘render_buffer_input’ is added to the operator.

enable_render_buffer_output

If True, an additional output port, named ‘render_buffer_output’ is added to the operator.

enable_camera_pose_output

If True, an additional output port, named ‘camera_pose_output’ is added to the operator.

font_path

File path for the font used for rendering text.

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

name

The name of the operator.

Notes

The tensors argument is used to specify the tensors to display. Each tensor is defined using a dictionary, that must, at minimum include a ‘name’ key that corresponds to a tensor found on the operator’s input. A ‘type’ key should also be provided to indicate the type of entry to display. The ‘type’ key will be one of {“color”, “color_lut”, “crosses”, “lines”, “lines_3d”, “line_strip”, “line_strip_3d”, “ovals”, “points”, “points_3d”, “rectangles”, “text”, “triangles”, “triangles_3d”, “depth_map”, “depth_map_color”, “unknown”}. The default type is “unknown” which will attempt to guess the corresponding type based on the tensor dimensions. Concrete examples are given below.

To show a single 2D RGB or RGBA image, use a list containing a single tensor of type ‘color’.

Copy

Copied!

            

            
tensors = [dict(name="video", type="color", opacity=1.0, priority=0)]
        

Here, the optional key opacity is used to scale the opacity of the tensor. The priority key is used to specify the render priority for layers. Layers with a higher priority will be rendered on top of those with a lower priority.

If we also had a “boxes” tensor representing rectangular bounding boxes, we could display them on top of the image like this.

Copy

Copied!

            

            
tensors = [
    dict(name="video", type="color", priority=0),
    dict(name="boxes", type="rectangles", color=[1.0, 0.0, 0.0], line_width=2, priority=1),
]
        

where the color and line_width keys specify the color and line width of the bounding box.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.holoviz._holoviz.HolovizOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.holoviz._holoviz.HolovizOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.InferenceOp

Bases: holoscan.core._core.Operator

Inference operator.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

DataMap
DataVecMap
OperatorType

class DataMap

Bases: pybind11_builtins.pybind11_object

Methods

get_map(self)
insert(self)

__init__(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → None

get_map(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → Dict[str, str]

insert(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → Dict[str, str]

class DataVecMap

Bases: pybind11_builtins.pybind11_object

Methods

get_map(self)
insert(self)

__init__(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → None

get_map(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → Dict[str, List[str]]

insert(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → Dict[str, List[str]]

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.inference._inference.InferenceOp, fragment: holoscan.core._core.Fragment, backend: str, allocator: holoscan.resources._resources.Allocator, inference_map: dict, model_path_map: dict, pre_processor_map: dict, device_map: dict = {}, backend_map: dict = {}, in_tensor_names: List[str] = [], out_tensor_names: List[str] = [], infer_on_cpu: bool = False, parallel_inference: bool = True, input_on_cuda: bool = True, output_on_cuda: bool = True, transmit_on_cuda: bool = True, enable_fp16: bool = False, is_engine_path: bool = False, cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'inference') → None

Inference operator.

Named inputs:

receivers: multi-receiver accepting nvidia::gxf::Tensor(s)

Any number of upstream ports may be connected to this receivers port. The operator will search across all messages for tensors matching those specified in in_tensor_names. These are the set of input tensors used by the models in inference_map.

Named outputs:

transmitter: nvidia::gxf::Tensor(s)

A message containing tensors corresponding to the inference results from all models will be emitted. The names of the tensors transmitted correspond to those in out_tensor_names.

Parameters

fragment

The fragment that the operator belongs to.

backend

Backend to use for inference. Set “trt” for TensorRT, “torch” for LibTorch and “onnxrt” for the ONNX runtime.

allocator

Memory allocator to use for the output.

inference_map

Tensor to model map.

model_path_map

Path to the ONNX model to be loaded.

pre_processor_map

Pre processed data to model map.

device_map

Mapping of model to GPU ID for inference.

backend_map: holoscan.operators.InferenceOp.DataMap, optional

Mapping of model to backend type for inference. Backend options: “trt” or “torch”

in_tensor_names

Input tensors.

out_tensor_names

Output tensors.

infer_on_cpu

Whether to run the computation on the CPU instead of GPU.

parallel_inference

Whether to enable parallel execution.

input_on_cuda

Whether the input buffer is on the GPU.

output_on_cuda

Whether the output buffer is on the GPU.

transmit_on_cuda

Whether to transmit the message on the GPU.

enable_fp16

Use 16-bit floating point computations.

is_engine_path

Whether the input model path mapping is for trt engine files

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.inference._inference.InferenceOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.inference._inference.InferenceOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.InferenceProcessorOp

Bases: holoscan.core._core.Operator

Holoinfer Processing operator.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

DataMap
DataVecMap
OperatorType

class DataMap

Bases: pybind11_builtins.pybind11_object

Methods

get_map(self)
insert(self)

__init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → None

get_map(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → Dict[str, str]

insert(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → Dict[str, str]

class DataVecMap

Bases: pybind11_builtins.pybind11_object

Methods

get_map(self)
insert(self)

__init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → None

get_map(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → Dict[str, List[str]]

insert(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → Dict[str, List[str]]

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, process_operations: dict = {}, processed_map: dict = {}, in_tensor_names: List[str] = [], out_tensor_names: List[str] = [], input_on_cuda: bool = False, output_on_cuda: bool = False, transmit_on_cuda: bool = False, disable_transmitter: bool = False, cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, config_path: str = '', name: str = 'postprocessor') → None

Holoinfer Processing operator.

Named inputs:

receivers: multi-receiver accepting nvidia::gxf::Tensor(s)

Any number of upstream ports may be connected to this receivers port. The operator will search across all messages for tensors matching those specified in in_tensor_names. These are the set of input tensors used by the processing operations specified in process_map.

Named outputs:

transmitter: nvidia::gxf::Tensor(s)

A message containing tensors corresponding to the processed results from operations will be emitted. The names of the tensors transmitted correspond to those in out_tensor_names.

Parameters

fragment

The fragment that the operator belongs to.

allocator

Memory allocator to use for the output.

process_operations

Operations in sequence on tensors.

processed_map

Input-output tensor mapping.

in_tensor_names

Names of input tensors in the order to be fed into the operator.

out_tensor_names

Names of output tensors in the order to be fed into the operator.

input_on_cuda

Whether the input buffer is on the GPU.

output_on_cuda

Whether the output buffer is on the GPU.

transmit_on_cuda

Whether to transmit the message on the GPU.

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

config_path

File path to the config file.

disable_transmitter

If True, disable the transmitter output port of the operator.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.NTV2Channel

Bases: pybind11_builtins.pybind11_object

Members:

NTV2_CHANNEL1

NTV2_CHANNEL2

NTV2_CHANNEL3

NTV2_CHANNEL4

NTV2_CHANNEL5

NTV2_CHANNEL6

NTV2_CHANNEL7

NTV2_CHANNEL8

NTV2_MAX_NUM_CHANNELS

NTV2_CHANNEL_INVALID

Attributes

name

value

NTV2_CHANNEL1 = <NTV2Channel.NTV2_CHANNEL1: 0>

NTV2_CHANNEL2 = <NTV2Channel.NTV2_CHANNEL2: 1>

NTV2_CHANNEL3 = <NTV2Channel.NTV2_CHANNEL3: 2>

NTV2_CHANNEL4 = <NTV2Channel.NTV2_CHANNEL4: 3>

NTV2_CHANNEL5 = <NTV2Channel.NTV2_CHANNEL5: 4>

NTV2_CHANNEL6 = <NTV2Channel.NTV2_CHANNEL6: 5>

NTV2_CHANNEL7 = <NTV2Channel.NTV2_CHANNEL7: 6>

NTV2_CHANNEL8 = <NTV2Channel.NTV2_CHANNEL8: 7>

NTV2_CHANNEL_INVALID = <NTV2Channel.NTV2_MAX_NUM_CHANNELS: 8>

NTV2_MAX_NUM_CHANNELS = <NTV2Channel.NTV2_MAX_NUM_CHANNELS: 8>

__init__(self: holoscan.operators.aja_source._aja_source.NTV2Channel, value: int) → None

property name

property value

class holoscan.operators.PingRxOp(fragment, *args, **kwargs)

Bases: holoscan.core.Operator

Simple receiver operator.

Named inputs:

in: any

A received value.

This is an example of a native operator with one input port. On each tick, it receives an integer from the “in” port.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(op_input, op_output, context)	Default implementation of compute
initialize()	Default implementation of initialize
setup(spec)	Default implementation of setup method.
start()	Default implementation of start
stop()	Default implementation of stop

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.core._core.Operator, arg0: object, arg1: holoscan::Fragment, *args, **kwargs) → None

Operator class.

Can be initialized with any number of Python positional and keyword arguments.

If a name keyword argument is provided, it must be a str and will be used to set the name of the Operator.

Condition classes will be added to self.conditions, Resource classes will be added to self.resources, and any other arguments will be cast from a Python argument type to a C++ Arg and stored in self.args. (For details on how the casting is done, see the py_object_to_arg utility). When a Condition or Resource is provided via a kwarg, it’s name will be automatically be updated to the name of the kwarg.

Parameters

fragment

The holoscan.core.Fragment (or holoscan.core.Application) to which this Operator will belong.

*args

Positional arguments.

**kwargs

Keyword arguments.

Raises

RuntimeError

If name kwarg is provided, but is not of str type. If multiple arguments of type Fragment are provided. If any other arguments cannot be converted to Arg type via py_object_to_arg.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(op_input, op_output, context)

Default implementation of compute

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize()

Default implementation of initialize

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(spec: holoscan.core._core.PyOperatorSpec)

Default implementation of setup method.

property spec

start()

Default implementation of start

stop()

Default implementation of stop

class holoscan.operators.PingTxOp(fragment, *args, **kwargs)

Bases: holoscan.core.Operator

Simple transmitter operator.

Named outputs:

out: int

An index value that increments by one on each call to compute. The starting value is 1.

On each tick, it transmits an integer to the “out” port.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(op_input, op_output, context)	Default implementation of compute
initialize()	Default implementation of initialize
setup(spec)	Default implementation of setup method.
start()	Default implementation of start
stop()	Default implementation of stop

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.core._core.Operator, arg0: object, arg1: holoscan::Fragment, *args, **kwargs) → None

Operator class.

Can be initialized with any number of Python positional and keyword arguments.

If a name keyword argument is provided, it must be a str and will be used to set the name of the Operator.

Condition classes will be added to self.conditions, Resource classes will be added to self.resources, and any other arguments will be cast from a Python argument type to a C++ Arg and stored in self.args. (For details on how the casting is done, see the py_object_to_arg utility). When a Condition or Resource is provided via a kwarg, it’s name will be automatically be updated to the name of the kwarg.

Parameters

fragment

The holoscan.core.Fragment (or holoscan.core.Application) to which this Operator will belong.

*args

Positional arguments.

**kwargs

Keyword arguments.

Raises

RuntimeError

If name kwarg is provided, but is not of str type. If multiple arguments of type Fragment are provided. If any other arguments cannot be converted to Arg type via py_object_to_arg.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(op_input, op_output, context)

Default implementation of compute

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize()

Default implementation of initialize

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(spec: holoscan.core._core.PyOperatorSpec)

Default implementation of setup method.

property spec

start()

Default implementation of start

stop()

Default implementation of stop

class holoscan.operators.SegmentationPostprocessorOp

Bases: holoscan.core._core.Operator

Operator carrying out post-processing operations on segmentation outputs.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Operator initialization method.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.segmentation_postprocessor._segmentation_postprocessor.SegmentationPostprocessorOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, in_tensor_name: str = '', network_output_type: str = 'softmax', data_format: str = 'hwc', cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'segmentation_postprocessor') → None

Operator carrying out post-processing operations on segmentation outputs.

Named inputs:

in_tensor: nvidia::gxf::Tensor

Expects a message containing a 32-bit floating point tensor with name in_tensor_name. The expected data layout of this tensor is HWC, NCHW or NHWC format as specified via data_format.

Named outputs:

out_tensor: nvidia::gxf::Tensor

Emits a message containing a tensor named “out_tensor” that contains the segmentation labels. This tensor will have unsigned 8-bit integer data type and shape (H, W, 1).

Parameters

fragment

The fragment that the operator belongs to.

allocator

Memory allocator to use for the output.

in_tensor_name

Name of the input tensor.

network_output_type

Network output type (e.g. ‘softmax’).

data_format

Data format of network output.

cuda_stream_pool

holoscan.resources.CudaStreamPool instance to allocate CUDA streams.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.core._core.Operator) → None

Operator initialization method.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.segmentation_postprocessor._segmentation_postprocessor.SegmentationPostprocessorOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.V4L2VideoCaptureOp

Bases: holoscan.core._core.Operator

Operator to get a video stream from a V4L2 source.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, device: str = '0', width: int = 0, height: int = 0, num_buffers: int = 4, pixel_format: str = 'auto', name: str = 'v4l2_video_capture') → None

Operator to get a video stream from a V4L2 source (e.g. Built-in HDMI capture card or USB camera)

https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/v4l2.html

Inputs a video stream from a V4L2 node, including USB cameras and HDMI IN.

• Input stream is on host. If no pixel format is specified in the yaml configuration file, the pixel format will be automatically selected. However, only AB24 and YUYV are then supported. If a pixel format is specified in the yaml file, then this format will be used. However, note that the operator then expects that this format can be encoded as RGBA32. If not, the behaviour is undefined.

• Output stream is on host. Always RGBA32 at this time.

Use holoscan.operators.FormatConverterOp to move data from the host to a GPU device.

Named output:

signal: nvidia::gxf::VideoBuffer

Emits a message containing a video buffer on the host with format GXF_VIDEO_FORMAT_RGBA.

Parameters

fragment

The fragment that the operator belongs to.

allocator : holoscan.resources.Allocator

Memory allocator to use for the output.

device

The device to target (e.g. “/dev/video0” for device 0)

width

Width of the video stream.

height

Height of the video stream.

num_buffers

Number of V4L2 buffers to use.

pixel_format

Video stream pixel format (little endian four character code (fourcc))

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec : holoscan.core.OperatorSpec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.VideoStreamRecorderOp

Bases: holoscan.core._core.Operator

Operator class to record the video stream to a file.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp, fragment: holoscan.core._core.Fragment, directory: str, basename: str, flush_on_tick: bool = False, name: str = 'recorder') → None

Operator class to record the video stream to a file.

Named inputs:

input: nvidia::gxf::Tensor

A message containing a video frame to serialize to disk.

Parameters

fragment

The fragment that the operator belongs to.

directory

Directory path for storing files.

basename

User specified file name without extension.

flush_on_tick

Flushes output buffer on every tick when True.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.

class holoscan.operators.VideoStreamReplayerOp

Bases: holoscan.core._core.Operator

Operator class to replay a video stream from a file.

Attributes

args	The list of arguments associated with the component.
conditions	Conditions associated with the operator.
description	YAML formatted string describing the operator.
fragment	The fragment that the operator belongs to.
id	The identifier of the component.
name	The name of the operator.
operator_type	The operator type.
resources	Resources associated with the operator.

spec

Methods

add_arg(*args, **kwargs)	Overloaded function.
compute(self, arg0, arg1, arg2)	Operator compute method.
initialize(self)	Initialize the operator.
setup(self, spec)	Define the operator specification.
start(self)	Operator start method.
stop(self)	Operator stop method.

OperatorType

class OperatorType

Bases: pybind11_builtins.pybind11_object

Members:

NATIVE

GXF

Attributes

name

value

GXF = <OperatorType.GXF: 1>

NATIVE = <OperatorType.NATIVE: 0>

__init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None

property name

property value

__init__(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp, fragment: holoscan.core._core.Fragment, directory: str, basename: str, batch_size: int = 1, ignore_corrupted_entities: bool = True, frame_rate: float = 1.0, realtime: bool = True, repeat: bool = False, count: int = 0, name: str = 'format_converter') → None

Operator class to replay a video stream from a file.

Named output:

output: nvidia::gxf::Tensor

A message containing a video frame deserialized from disk.

Parameters

fragment

The fragment that the operator belongs to.

directory

Directory path for reading files from.

basename

User specified file name without extension.

batch_size

Number of entities to read and publish for one tick.

ignore_corrupted_entities

If an entity could not be deserialized, it is ignored by default; otherwise a failure is generated.

frame_rate

Frame rate to replay. If zero value is specified, it follows timings in timestamps.

realtime

Playback video in realtime, based on frame_rate or timestamps.

repeat

Repeat video stream in a loop.

count

Number of frame counts to playback. If zero value is specified, it is ignored. If the count is less than the number of frames in the video, it would finish early.

name

The name of the operator.

add_arg(*args, **kwargs)

Overloaded function.

1. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None

Add an argument to the component.

2. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None

Add a list of arguments to the component.

3. add_arg(self: holoscan.core._core.Operator, **kwargs) -> None

Add arguments to the component via Python kwargs.

4. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None

5. add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None

Add a condition or resource to the Operator.

This can be used to add a condition or resource to an operator after it has already been constructed.

Parameters

arg

The condition or resource to add.

property args

The list of arguments associated with the component.

Returns

arglist

compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None

Operator compute method. This method defines the primary computation to be executed by the operator.

property conditions

Conditions associated with the operator.

property description

YAML formatted string describing the operator.

property fragment

The fragment that the operator belongs to.

Returns

name

property id

The identifier of the component.

The identifier is initially set to -1, and will become a valid value when the component is initialized.

With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.

Returns

id

initialize(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp) → None

Initialize the operator.

This method is called only once when the operator is created for the first time, and uses a light-weight initialization.

property name

The name of the operator.

Returns

name

property operator_type

The operator type.

holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.

property resources

Resources associated with the operator.

setup(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp, spec: holoscan.core._core.OperatorSpec) → None

Define the operator specification.

Parameters

spec

The operator specification.

property spec

start(self: holoscan.core._core.Operator) → None

Operator start method.

stop(self: holoscan.core._core.Operator) → None

Operator stop method.