holoscan.operators
This module provides a Python API to underlying C++ API Operators.
<a href="#holoscan.operators.AJASourceOp">holoscan.operators.AJASourceOp</a> |
Operator to get a video stream from an AJA capture card. |
<a href="#holoscan.operators.BayerDemosaicOp">holoscan.operators.BayerDemosaicOp</a> |
Bayer Demosaic operator. |
<a href="#holoscan.operators.FormatConverterOp">holoscan.operators.FormatConverterOp</a> |
Format conversion operator. |
<a href="#holoscan.operators.HolovizOp">holoscan.operators.HolovizOp</a> (fragment[, ...]) |
Holoviz visualization operator using Holoviz module. |
<a href="#holoscan.operators.InferenceOp">holoscan.operators.InferenceOp</a> |
Inference operator. |
<a href="#holoscan.operators.InferenceProcessorOp">holoscan.operators.InferenceProcessorOp</a> |
Holoinfer Processing operator. |
<a href="#holoscan.operators.PingRxOp">holoscan.operators.PingRxOp</a> (fragment, *args, ...) |
Simple receiver operator. |
<a href="#holoscan.operators.PingTxOp">holoscan.operators.PingTxOp</a> (fragment, *args, ...) |
Simple transmitter operator. |
<a href="#holoscan.operators.SegmentationPostprocessorOp">holoscan.operators.SegmentationPostprocessorOp</a> |
Operator carrying out post-processing operations on segmentation outputs. |
<a href="#holoscan.operators.V4L2VideoCaptureOp">holoscan.operators.V4L2VideoCaptureOp</a> |
Operator to get a video stream from a V4L2 source. |
<a href="#holoscan.operators.VideoStreamRecorderOp">holoscan.operators.VideoStreamRecorderOp</a> |
Operator class to record a video stream to a file. |
<a href="#holoscan.operators.VideoStreamReplayerOp">holoscan.operators.VideoStreamReplayerOp</a> |
Operator class to replay a video stream from a file. |
- class holoscan.operators.AJASourceOp
Bases:
holoscan.core._core.Operator
Operator to get a video stream from an AJA capture card.
==Named Inputs==
- overlay_buffer_inputnvidia::gxf::VideoBuffer (optional)
The operator does not require a message on this input port in order for
compute
to be called. If a message is found, andenable_overlay
isTrue
, the image will be mixed with the image captured by the AJA card. Ifenable_overlay
isFalse
, any message on this port will be ignored.==Named Outputs==
- video_buffer_outputnvidia::gxf::VideoBuffer
- overlay_buffer_outputnvidia::gxf::VideoBuffer (optional)
The output video frame from the AJA capture card. If
overlay_rdma
isTrue
, this video buffer will be on the device, otherwise it will be in pinned host memory.This output port will only emit a video buffer when
enable_overlay
isTrue
. Ifoverlay_rdma
isTrue
, this video buffer will be on the device, otherwise it will be in pinned host memory.- Parameters
- fragment
- device
- channel
- width
- height
- framerate
- rdma
- enable_overlay
- overlay_channel
- overlay_rdma
- name
The fragment that the operator belongs to.
The device to target (e.g., “0” for device 0). Default value is
"0"
.The camera
NTV2Channel
to use for output (e.g.,NTV2Channel.NTV2_CHANNEL1
(0
) or “NTV2_CHANNEL1” (in YAML) for the first channel). Default value isNTV2Channel.NTV2_CHANNEL1
("NTV2_CHANNEL1"
in YAML).Width of the video stream. Default value is
1920
.Height of the video stream. Default value is
1080
.Frame rate of the video stream. Default value is
60
.Boolean indicating whether RDMA is enabled. Default value is
False
("false"
in YAML).Boolean indicating whether a separate overlay channel is enabled. Default value is
False
("false"
in YAML).The camera NTV2Channel to use for overlay output. Default value is
NTV2Channel.NTV2_CHANNEL2
("NTV2_CHANNEL2"
in YAML).Boolean indicating whether RDMA is enabled for the overlay. Default value is
False
("false"
in YAML).The name of the operator. Default value is
"aja_source"
.
Attributes
<a href="#holoscan.operators.AJASourceOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.AJASourceOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.AJASourceOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.AJASourceOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.AJASourceOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.AJASourceOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.AJASourceOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.AJASourceOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.AJASourceOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.AJASourceOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.AJASourceOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.AJASourceOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.AJASourceOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.AJASourceOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.AJASourceOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.AJASourceOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.aja_source._aja_source.AJASourceOp, fragment: holoscan.core._core.Fragment, device: str = '0', channel: holoscan.operators.aja_source._aja_source.NTV2Channel = <NTV2Channel.NTV2_CHANNEL1: 0>, width: int = 1920, height: int = 1080, framerate: int = 60, rdma: bool = False, enable_overlay: bool = False, overlay_channel: holoscan.operators.aja_source._aja_source.NTV2Channel = <NTV2Channel.NTV2_CHANNEL2: 1>, overlay_rdma: bool = True, name: str = 'aja_source') → None
Operator to get a video stream from an AJA capture card.
==Named Inputs==
- overlay_buffer_inputnvidia::gxf::VideoBuffer (optional)
The operator does not require a message on this input port in order for
compute
to be called. If a message is found, andenable_overlay
isTrue
, the image will be mixed with the image captured by the AJA card. Ifenable_overlay
isFalse
, any message on this port will be ignored.==Named Outputs==
- video_buffer_outputnvidia::gxf::VideoBuffer
- overlay_buffer_outputnvidia::gxf::VideoBuffer (optional)
The output video frame from the AJA capture card. If
overlay_rdma
isTrue
, this video buffer will be on the device, otherwise it will be in pinned host memory.This output port will only emit a video buffer when
enable_overlay
isTrue
. Ifoverlay_rdma
isTrue
, this video buffer will be on the device, otherwise it will be in pinned host memory.- Parameters
- fragment
- device
- channel
- width
- height
- framerate
- rdma
- enable_overlay
- overlay_channel
- overlay_rdma
- name
The fragment that the operator belongs to.
The device to target (e.g., “0” for device 0). Default value is
"0"
.The camera
NTV2Channel
to use for output (e.g.,NTV2Channel.NTV2_CHANNEL1
(0
) or “NTV2_CHANNEL1” (in YAML) for the first channel). Default value isNTV2Channel.NTV2_CHANNEL1
("NTV2_CHANNEL1"
in YAML).Width of the video stream. Default value is
1920
.Height of the video stream. Default value is
1080
.Frame rate of the video stream. Default value is
60
.Boolean indicating whether RDMA is enabled. Default value is
False
("false"
in YAML).Boolean indicating whether a separate overlay channel is enabled. Default value is
False
("false"
in YAML).The camera NTV2Channel to use for overlay output. Default value is
NTV2Channel.NTV2_CHANNEL2
("NTV2_CHANNEL2"
in YAML).Boolean indicating whether RDMA is enabled for the overlay. Default value is
False
("false"
in YAML).The name of the operator. Default value is
"aja_source"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.aja_source._aja_source.AJASourceOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.aja_source._aja_source.AJASourceOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.BayerDemosaicOp
Bases:
holoscan.core._core.Operator
Bayer Demosaic operator.
==Named Inputs==
- receivernvidia::gxf::Tensor or nvidia::gxf::VideoBuffer
The input video frame to process. If the input is a VideoBuffer it must be an 8-bit unsigned grayscale video (nvidia::gxf::VideoFormat::GXF_VIDEO_FORMAT_GRAY). The video buffer may be in either host or device memory (a host->device copy is performed if needed). If a video buffer is not found, the input port message is searched for a tensor with the name specified by
in_tensor_name
. This must be a device tensor in either 8-bit or 16-bit unsigned integer format.==Named Outputs==
- transmitternvidia::gxf::Tensor
The output video frame after demosaicing. This will be a 3-channel RGB image if
alpha_value
isTrue
, otherwise it will be a 4-channel RGBA image. The data type will be either 8-bit or 16-bit unsigned integer (matching the bit depth of the input). The name of the tensor that is output is controlled byout_tensor_name
.- Parameters
- fragment
- pool
- cuda_stream_pool
- in_tensor_name
- out_tensor_name
- interpolation_mode
NPPI_INTER_UNDEFINED (
0
): Undefined filtering interpolation mode.NPPI_INTER_NN (
1
): Nearest neighbor filtering.NPPI_INTER_LINEAR (
2
): Linear interpolation.NPPI_INTER_CUBIC (
4
): Cubic interpolation.NPPI_INTER_CUBIC2P_BSPLINE (
5
): Two-parameter cubic filter (B=1, C=0)NPPI_INTER_CUBIC2P_CATMULLROM (
6
): Two-parameter cubic filter (B=0, C=1/2)NPPI_INTER_CUBIC2P_B05C03 (
7
): Two-parameter cubic filter (B=1/2, C=3/10)NPPI_INTER_SUPER (
8
): Super sampling.NPPI_INTER_LANCZOS (
16
): Lanczos filtering.NPPI_INTER_LANCZOS3_ADVANCED (
17
): Generic Lanczos filtering with order 3.NPPI_SMOOTH_EDGE (
0x8000000
): Smooth edge filtering.- bayer_grid_pos
NPPI_BAYER_BGGR (
0
): Default registration position BGGR.NPPI_BAYER_RGGB (
1
): Registration position RGGB.NPPI_BAYER_GBRG (
2
): Registration position GBRG.NPPI_BAYER_GRBG (
3
): Registration position GRBG.- generate_alpha
- alpha_value
- name
The fragment that the operator belongs to.
Memory pool allocator used by the operator.
holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the input tensor. Default value is
""
(empty string).The name of the output tensor. Default value is
""
(empty string).The interpolation model to be used for demosaicing. Values available at: https://docs.nvidia.com/cuda/npp/nppdefs.html?highlight=Two%20parameter%20cubic%20filter#c.NppiInterpolationMode
Default value is
0
(NPPI_INTER_UNDEFINED).The Bayer grid position. Values available at: https://docs.nvidia.com/cuda/npp/nppdefs.html?highlight=Two%20parameter%20cubic%20filter#c.NppiBayerGridPosition
Default value is
2
(NPPI_BAYER_GBRG).Generate alpha channel. Default value is
False
.Alpha value to be generated if
generate_alpha
is set toTrue
. Default value is255
.The name of the operator. Default value is
"bayer_demosaic"
.
Attributes
<a href="#holoscan.operators.BayerDemosaicOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.BayerDemosaicOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.BayerDemosaicOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.BayerDemosaicOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.BayerDemosaicOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.BayerDemosaicOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.BayerDemosaicOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.BayerDemosaicOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.BayerDemosaicOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.BayerDemosaicOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.BayerDemosaicOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.BayerDemosaicOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.BayerDemosaicOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.BayerDemosaicOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.BayerDemosaicOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.BayerDemosaicOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.bayer_demosaic._bayer_demosaic.BayerDemosaicOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.FormatConverterOp
Bases:
holoscan.core._core.Operator
Format conversion operator.
==Named Inputs==
- source_videonvidia::gxf::Tensor or nvidia::gxf::VideoBuffer
The input video frame to process. If the input is a VideoBuffer it must be in format GXF_VIDEO_FORMAT_RGBA, GXF_VIDEO_FORMAT_RGB or GXF_VIDEO_FORMAT_NV12. This video buffer may be in either host or device memory (a host->device copy is performed if needed). If a video buffer is not found, the input port message is searched for a tensor with the name specified by
in_tensor_name
. This must be a device tensor in one of several supported formats (unsigned 8-bit int or float32 graycale, unsigned 8-bit int RGB or RGBA, YUV420 or NV12).==Named Outputs==
- tensornvidia::gxf::Tensor
The output video frame after processing. The shape, data type and number of channels of this output tensor will depend on the specific parameters that were set for this operator. The name of the Tensor transmitted on this port is determined by
out_tensor_name
.- Parameters
- fragment
- pool
- out_dtype
"rgb888"
"uint8"
"float32"
"rgba8888"
"yuv420"
"nv12"
- in_dtype
"rgb888"
"uint8"
"float32"
"rgba8888"
"yuv420"
"nv12"
- in_tensor_name
- out_tensor_name
- scale_min
- scale_max
- alpha_value
- resize_height
- resize_width
- resize_mode
NPPI_INTER_UNDEFINED (
0
): Undefined filtering interpolation mode.NPPI_INTER_NN (
1
): Nearest neighbor filtering.NPPI_INTER_LINEAR (
2
): Linear interpolation.NPPI_INTER_CUBIC (
4
): Cubic interpolation.NPPI_INTER_CUBIC2P_BSPLINE (
5
): Two-parameter cubic filter (B=1, C=0)NPPI_INTER_CUBIC2P_CATMULLROM (
6
): Two-parameter cubic filter (B=0, C=1/2)NPPI_INTER_CUBIC2P_B05C03 (
7
): Two-parameter cubic filter (B=1/2, C=3/10)NPPI_INTER_SUPER (
8
): Super sampling.NPPI_INTER_LANCZOS (
16
): Lanczos filtering.NPPI_INTER_LANCZOS3_ADVANCED (
17
): Generic Lanczos filtering with order 3.NPPI_SMOOTH_EDGE (
0x8000000
): Smooth edge filtering.- channel_order
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Memory pool allocator used by the operator.
Destination data type. The available options are:
Source data type. The available options are:
The name of the input tensor. Default value is
""
(empty string).The name of the output tensor. Default value is
""
(empty string).Output will be clipped to this minimum value. Default value is
0.0
.Output will be clipped to this maximum value. Default value is
1.0
.Unsigned integer in range [0, 255], indicating the alpha channel value to use when converting from RGB to RGBA. Default value is
255
.Desired height for the (resized) output. Height will be unchanged if
resize_height
is0
. Default value is0
.Desired width for the (resized) output. Width will be unchanged if
resize_width
is0
. Default value is0
.Resize mode enum value corresponding to NPP’s NppiInterpolationMode. Values available at: https://docs.nvidia.com/cuda/npp/nppdefs.html?highlight=Two%20parameter%20cubic%20filter#c.NppiInterpolationMode
Default value is
0
(NPPI_INTER_UNDEFINED) which would be equivalent to4
(NPPI_INTER_CUBIC).Sequence of integers describing how channel values are permuted. Default value is
[0, 1, 2]
for 3-channel images and[0, 1, 2, 3]
for 4-channel images.holoscan.resources.CudaStreamPool instance to allocate CUDA streams. Default value is
None
.The name of the operator. Default value is
"format_converter"
.
Attributes
<a href="#holoscan.operators.FormatConverterOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.FormatConverterOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.FormatConverterOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.FormatConverterOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.FormatConverterOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.FormatConverterOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.FormatConverterOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.FormatConverterOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.FormatConverterOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.FormatConverterOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.FormatConverterOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.FormatConverterOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.FormatConverterOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.FormatConverterOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.FormatConverterOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.FormatConverterOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.format_converter._format_converter.FormatConverterOp, fragment: holoscan.core._core.Fragment, pool: holoscan.resources._resources.Allocator, out_dtype: str, in_dtype: str = '', in_tensor_name: str = '', out_tensor_name: str = '', scale_min: float = 0.0, scale_max: float = 1.0, alpha_value: int = 255, resize_height: int = 0, resize_width: int = 0, resize_mode: int = 0, out_channel_order: List[int] = [], cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'format_converter') → None
Format conversion operator.
==Named Inputs==
- source_videonvidia::gxf::Tensor or nvidia::gxf::VideoBuffer
The input video frame to process. If the input is a VideoBuffer it must be in format GXF_VIDEO_FORMAT_RGBA, GXF_VIDEO_FORMAT_RGB or GXF_VIDEO_FORMAT_NV12. This video buffer may be in either host or device memory (a host->device copy is performed if needed). If a video buffer is not found, the input port message is searched for a tensor with the name specified by
in_tensor_name
. This must be a device tensor in one of several supported formats (unsigned 8-bit int or float32 graycale, unsigned 8-bit int RGB or RGBA, YUV420 or NV12).==Named Outputs==
- tensornvidia::gxf::Tensor
The output video frame after processing. The shape, data type and number of channels of this output tensor will depend on the specific parameters that were set for this operator. The name of the Tensor transmitted on this port is determined by
out_tensor_name
.- Parameters
- fragment
- pool
- out_dtype
"rgb888"
"uint8"
"float32"
"rgba8888"
"yuv420"
"nv12"
- in_dtype
"rgb888"
"uint8"
"float32"
"rgba8888"
"yuv420"
"nv12"
- in_tensor_name
- out_tensor_name
- scale_min
- scale_max
- alpha_value
- resize_height
- resize_width
- resize_mode
NPPI_INTER_UNDEFINED (
0
): Undefined filtering interpolation mode.NPPI_INTER_NN (
1
): Nearest neighbor filtering.NPPI_INTER_LINEAR (
2
): Linear interpolation.NPPI_INTER_CUBIC (
4
): Cubic interpolation.NPPI_INTER_CUBIC2P_BSPLINE (
5
): Two-parameter cubic filter (B=1, C=0)NPPI_INTER_CUBIC2P_CATMULLROM (
6
): Two-parameter cubic filter (B=0, C=1/2)NPPI_INTER_CUBIC2P_B05C03 (
7
): Two-parameter cubic filter (B=1/2, C=3/10)NPPI_INTER_SUPER (
8
): Super sampling.NPPI_INTER_LANCZOS (
16
): Lanczos filtering.NPPI_INTER_LANCZOS3_ADVANCED (
17
): Generic Lanczos filtering with order 3.NPPI_SMOOTH_EDGE (
0x8000000
): Smooth edge filtering.- channel_order
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Memory pool allocator used by the operator.
Destination data type. The available options are:
Source data type. The available options are:
The name of the input tensor. Default value is
""
(empty string).The name of the output tensor. Default value is
""
(empty string).Output will be clipped to this minimum value. Default value is
0.0
.Output will be clipped to this maximum value. Default value is
1.0
.Unsigned integer in range [0, 255], indicating the alpha channel value to use when converting from RGB to RGBA. Default value is
255
.Desired height for the (resized) output. Height will be unchanged if
resize_height
is0
. Default value is0
.Desired width for the (resized) output. Width will be unchanged if
resize_width
is0
. Default value is0
.Resize mode enum value corresponding to NPP’s NppiInterpolationMode. Values available at: https://docs.nvidia.com/cuda/npp/nppdefs.html?highlight=Two%20parameter%20cubic%20filter#c.NppiInterpolationMode
Default value is
0
(NPPI_INTER_UNDEFINED) which would be equivalent to4
(NPPI_INTER_CUBIC).Sequence of integers describing how channel values are permuted. Default value is
[0, 1, 2]
for 3-channel images and[0, 1, 2, 3]
for 4-channel images.holoscan.resources.CudaStreamPool instance to allocate CUDA streams. Default value is
None
.The name of the operator. Default value is
"format_converter"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.format_converter._format_converter.FormatConverterOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.format_converter._format_converter.FormatConverterOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.HolovizOp(fragment, allocator=None, receivers=(), tensors=(), color_lut=(), window_title='Holoviz', display_name='DP-0', width=1920, height=1080, framerate=60, use_exclusive_display=False, fullscreen=False, headless=False, enable_render_buffer_input=False, enable_render_buffer_output=False, enable_camera_pose_output=False, font_path='', cuda_stream_pool=None, name='holoviz_op')
Bases:
holoscan.operators.holoviz._holoviz.HolovizOp
Holoviz visualization operator using Holoviz module.
This is a Vulkan-based visualizer.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor and/or nvidia::gxf::VideoBuffer
- input_specslist[holoscan.operators.HolovizOp.InputSpec] (optional)
- render_buffer_inputnvidia::gxf::VideoBuffer (optional)
Any number of upstream ports may be connected to this
receivers
port. This port can accept either VideoBuffers or Tensors. These inputs can be in either host or device memory. Each tensor or video buffer will result in a layer. The operator autodetects the layer type for certain input types (e.g. a video buffer will result in an image layer). For other input types or more complex use cases, input specifications can be provided either at initialization time as a parameter or dynamically at run time (viainput_specs
). On each call tocompute
, tensors corresponding to all names specified in thetensors
parameter must be found or an exception will be raised. Any extra, named tensors not present in thetensors
parameter specification (or optional, dynamicinput_specs
input) will be ignored.A list of
InputSpec
objects. This port can be used to dynamically update the overlay specification at run time. No inputs are required on this port in order for the operator tocompute
.An empty render buffer can optionally be provided. The video buffer must have format GXF_VIDEO_FORMAT_RGBA and be in device memory. This input port only exists if
enable_render_buffer_input
was set toTrue
, in which casecompute
will only be called when a message arrives on this input.==Named Outputs==
- render_buffer_outputnvidia::gxf::VideoBuffer (optional)
- camera_pose_outputstd::array<float, 16> (optional)
Output for a filled render buffer. If an input render buffer is specified, it is using that one, else it allocates a new buffer. The video buffer will have format GXF_VIDEO_FORMAT_RGBA and will be in device memory. This output is useful for offline rendering or headless mode. This output port only exists if
enable_render_buffer_output
was set toTrue
.The camera pose. The parameters returned represent the values of a 4x4 row major projection matrix. This output port only exists if
enable_camera_pose_output
was set toTrue
.- Parameters
- fragment
- allocator
- receivers
- tensors
- color_lut
- window_title
- display_name
- width
- height
- framerate
- use_exclusive_display
- fullscreen
- headless
- enable_render_buffer_input
- enable_render_buffer_output
- enable_camera_pose_output
- font_path
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Allocator used to allocate render buffer output. If
None
, will default toholoscan.core.UnboundedAllocator
.List of input receivers.
List of input tensors. Each tensor is defined by a dictionary where the
"name"
key must correspond to a tensor sent to the operator’s input. See the notes section below for further details on how the tensor dictionary is defined.Color lookup table for tensors of type
color_lut
. Should be shape(n_colors, 4)
.Title on window canvas. Default value is
"Holoviz"
.In exclusive mode, name of display to use as shown with xrandr. Default value is
"DP-0"
.Window width or display resolution width if in exclusive or fullscreen mode. Default value is
1920
.Window height or display resolution width if in exclusive or fullscreen mode. Default value is
1080
.Display framerate in Hz if in exclusive mode. Default value is
60.0
.Enable exclusive display. Default value is
False
.Enable fullscreen window. Default value is
False
.Enable headless mode. No window is opened, the render buffer is output to port
render_buffer_output
. Default value isFalse
.If
True
, an additional input port, named"render_buffer_input"
is added to the operator. Default value isFalse
.If
True
, an additional output port, named"render_buffer_output"
is added to the operator. Default value isFalse
.If
True
, an additional output port, named"camera_pose_output"
is added to the operator. Default value isFalse
.File path for the font used for rendering text. Default value is
""
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"holoviz_op"
.
Notes
The
tensors
argument is used to specify the tensors to display. Each tensor is defined using a dictionary, that must, at minimum include a ‘name’ key that corresponds to a tensor found on the operator’s input. A ‘type’ key should also be provided to indicate the type of entry to display. The ‘type’ key will be one of {"color"
,"color_lut"
,"crosses"
,"lines"
,"lines_3d"
,"line_strip"
,"line_strip_3d"
,"ovals"
,"points"
,"points_3d"
,"rectangles"
,"text"
,"triangles"
,"triangles_3d"
,"depth_map"
,"depth_map_color"
,"unknown"
}. The default type is"unknown"
which will attempt to guess the corresponding type based on the tensor dimensions. Concrete examples are given below.To show a single 2D RGB or RGBA image, use a list containing a single tensor of type
"color"
.tensors = [dict(name="video", type="color", opacity=1.0, priority=0)]
Here, the optional key
opacity
is used to scale the opacity of the tensor. Thepriority
key is used to specify the render priority for layers. Layers with a higher priority will be rendered on top of those with a lower priority.If we also had a
"boxes"`
tensor representing rectangular bounding boxes, we could display them on top of the image like this.tensors = [ dict(name="video", type="color", priority=0), dict(name="boxes", type="rectangles", color=[1.0, 0.0, 0.0], line_width=2, priority=1), ]
where the
color
andline_width
keys specify the color and line width of the bounding box.The details of the dictionary is as follows:
name: name of the tensor containing the input data to display
type:
str
type: input type (default
"unknown"
)type:
str
possible values:
unknown: unknown type, the operator tries to guess the type by inspecting the tensor.
color: RGB or RGBA color 2d image.
color_lut: single channel 2d image, color is looked up.
points: point primitives, one coordinate (x, y) per primitive.
lines: line primitives, two coordinates (x0, y0) and (x1, y1) per primitive.
line_strip: line strip primitive, a line primitive i is defined by each coordinate (xi, yi) and the following (xi+1, yi+1).
triangles: triangle primitive, three coordinates (x0, y0), (x1, y1) and (x2, y2) per primitive.
crosses: cross primitive, a cross is defined by the center coordinate and the size (xi, yi, si).
rectangles: axis aligned rectangle primitive, each rectangle is defined by two coordinates (xi, yi) and (xi+1, yi+1).
ovals: oval primitive, an oval primitive is defined by the center coordinate and the axis sizes (xi, yi, sxi, syi).
text: text is defined by the top left coordinate and the size (x, y, s) per string, text strings are defined by InputSpec member text.
depth_map: single channel 2d array where each element represents a depth value. The data is rendered as a 3d object using points, lines or triangles. The color for the elements can be specified through
depth_map_color
. Supported format: 8-bit unsigned normalized format that has a single 8-bit depth component.depth_map_color: RGBA 2d image, same size as the depth map. One color value for each element of the depth map grid. Supported format: 32-bit unsigned normalized format that has an 8-bit R component in byte 0, an 8-bit G component in byte 1, an 8-bit B component in byte 2, and an 8-bit A component in byte 3.
opacity: layer opacity, 1.0 is fully opaque, 0.0 is fully transparent (default:
1.0
)type:
float
- priority: layer priority, determines the render order, layers with higher priority
values are rendered on top of layers with lower priority values (default:
0
)type:
int
color: RGBA color of rendered geometry (default:
[1.f, 1.f, 1.f, 1.f]
)type:
List[float]
line_width: line width for geometry made of lines (default:
1.0
)type:
float
point_size: point size for geometry made of points (default:
1.0
)type:
float
text: array of text strings, used when
type
is text (default:[]
)type:
List[str]
depth_map_render_mode: depth map render mode (default:
points
)type:
str
possible values:
points: render as points
lines: render as lines
triangles: render as triangles
Displaying Color Images
Image data can either be on host or device (GPU). Multiple image formats are supported
R 8 bit unsigned
R 16 bit unsigned
R 16 bit float
R 32 bit unsigned
R 32 bit float
RGB 8 bit unsigned
BGR 8 bit unsigned
RGBA 8 bit unsigned
BGRA 8 bit unsigned
RGBA 16 bit unsigned
RGBA 16 bit float
RGBA 32 bit float
When the
type
parameter is set tocolor_lut
the final color is looked up using the values from thecolor_lut
parameter. For color lookups these image formats are supportedR 8 bit unsigned
R 16 bit unsigned
R 32 bit unsigned
Drawing Geometry
In all cases,
x
andy
are normalized coordinates in the range[0, 1]
. Thex
andy
correspond to the horizontal and vertical axes of the display, respectively. The origin(0, 0)
is at the top left of the display. Geometric primitives outside of the visible area are clipped. Coordinate arrays are expected to have the shape(N, C)
whereN
is the coordinate count andC
is the component count for each coordinate.Points are defined by a
(x, y)
coordinate pair.Lines are defined by a set of two
(x, y)
coordinate pairs.Lines strips are defined by a sequence of
(x, y)
coordinate pairs. The first two coordinates define the first line, each additional coordinate adds a line connecting to the previous coordinate.Triangles are defined by a set of three
(x, y)
coordinate pairs.Crosses are defined by
(x, y, size)
tuples.size
specifies the size of the cross in thex
direction and is optional, if omitted it’s set to0.05
. The size in they
direction is calculated using the aspect ratio of the window to make the crosses square.Rectangles (bounding boxes) are defined by a pair of 2-tuples defining the upper-left and lower-right coordinates of a box:
(x1, y1), (x2, y2)
.Ovals are defined by
(x, y, size_x, size_y)
tuples.size_x
andsize_y
are optional, if omitted they are set to0.05
.Texts are defined by
(x, y, size)
tuples.size
specifies the size of the text iny
direction and is optional, if omitted it’s set to0.05
. The size in thex
direction is calculated using the aspect ratio of the window. The index of each coordinate references a text string from thetext
parameter and the index is clamped to the size of the text array. For example, if there is one item set for thetext
parameter, e.g.text=["my_text"]
and three coordinates, thenmy_text
is rendered three times. Iftext=["first text", "second text"]
and three coordinates are specified, thenfirst text
is rendered at the first coordinate,second text
at the second coordinate and thensecond text
again at the third coordinate. Thetext
string array is fixed and can’t be changed after initialization. To hide text which should not be displayed, specify coordinates greater than(1.0, 1.0)
for the text item, the text is then clipped away.3D Points are defined by a
(x, y, z)
coordinate tuple.3D Lines are defined by a set of two
(x, y, z)
coordinate tuples.3D Lines strips are defined by a sequence of
(x, y, z)
coordinate tuples. The first two coordinates define the first line, each additional coordinate adds a line connecting to the previous coordinate.3D Triangles are defined by a set of three
(x, y, z)
coordinate tuples.
Displaying Depth Maps
When
type
isdepth_map
the provided data is interpreted as a rectangular array of depth values. Additionally a 2d array with a color value for each point in the grid can be specified by settingtype
todepth_map_color
.The type of geometry drawn can be selected by setting
depth_map_render_mode
.Depth maps are rendered in 3D and support camera movement. The camera is controlled using the mouse:
Orbit (LMB)
Pan (LMB + CTRL | MMB)
Dolly (LMB + SHIFT | RMB | Mouse wheel)
Look Around (LMB + ALT | LMB + CTRL + SHIFT)
Zoom (Mouse wheel + SHIFT)
Output
By default a window is opened to display the rendering, but the extension can also be run in headless mode with the
headless
parameter.Using a display in exclusive mode is also supported with the
use_exclusive_display
parameter. This reduces the latency by avoiding the desktop compositor.The rendered framebuffer can be output to
render_buffer_output
.
Attributes
<a href="#holoscan.operators.HolovizOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.HolovizOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.HolovizOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.HolovizOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.HolovizOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.HolovizOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.HolovizOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.HolovizOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.HolovizOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.HolovizOp.InputSpec">InputSpec</a>
InputSpec for the HolovizOp operator. <a href="#holoscan.operators.HolovizOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.HolovizOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.HolovizOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.HolovizOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.HolovizOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.HolovizOp.stop">stop</a>
(self)Operator stop method. DepthMapRenderMode InputType OperatorType - class DepthMapRenderMode
Bases:
pybind11_builtins.pybind11_object
Members:
POINTS
LINES
TRIANGLES
Attributes
<a href="#holoscan.operators.HolovizOp.DepthMapRenderMode.name">name</a>
value - LINES = <DepthMapRenderMode.LINES: 1>
- POINTS = <DepthMapRenderMode.POINTS: 0>
- TRIANGLES = <DepthMapRenderMode.TRIANGLES: 2>
- __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.DepthMapRenderMode, value: int) → None
- property name
- property value
- class InputSpec
Bases:
pybind11_builtins.pybind11_object
InputSpec for the HolovizOp operator.
- Parameters
- tensor_name
- type
The tensor name for this input.
The type of data that this tensor represents.
Attributes
type (holoscan.operators.HolovizOp.InputType) The type of data that this tensor represents. opacity (float) The opacity of the object. Must be in range [0.0, 1.0] where 1.0 is fully opaque. priority (int) Layer priority, determines the render order. Layers with higher priority values are rendered on top of layers with lower priority. color (4-tuple of float) RGBA values in range [0.0, 1.0] for rendered geometry. line_width (float) Line width for geometry made of lines. point_size (float) Point size for geometry made of points. text (sequence of str) Sequence of strings used when type is HolovizOp.InputType.TEXT. depth_map_render_mode (holoscan.operators.HolovizOp.DepthMapRenderMode) The depth map render mode. Used only if type is HolovizOp.InputType.DEPTH_MAP or HolovizOp.InputType.DEPTH_MAP_COLOR. views (list of HolovizOp.InputSpec.View) Sequence of layer views. By default a layer will fill the whole window. When using a view, the layer can be placed freely within the window. When multiple views are specified, the layer is drawn multiple times using the specified layer views. Methods
<a href="#holoscan.operators.HolovizOp.InputSpec.View">View</a>
View for the InputSpec of a HolovizOp operator. <a href="#holoscan.operators.HolovizOp.InputSpec.description">description</a>
(self)- Returns
- class View
Bases:
pybind11_builtins.pybind11_object
View for the InputSpec of a HolovizOp operator.
Notes
Layers can also be placed in 3D space by specifying a 3D transformation matrix. Note that for geometry layers there is a default matrix which allows coordinates in the range of [0 … 1] instead of the Vulkan [-1 … 1] range. When specifying a matrix for a geometry layer, this default matrix is overwritten.
When multiple views are specified, the layer is drawn multiple times using the specified layer views.
It’s possible to specify a negative term for height, which flips the image. When using a negative height, one should also adjust the y value to point to the lower left corner of the viewport instead of the upper left corner.
Attributes
offset_x, offset_y (float) Offset of top-left corner of the view. (0, 0) is the upper left and (1, 1) is the lower right. width (float) Normalized width (range [0.0, 1.0]). height (float) Normalized height (range [0.0, 1.0]). matrix (sequence of float) 16-elements representing a 4x4 transformation matrix. - __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec.View) → None
View for the InputSpec of a HolovizOp operator.
Notes
Layers can also be placed in 3D space by specifying a 3D transformation matrix. Note that for geometry layers there is a default matrix which allows coordinates in the range of [0 … 1] instead of the Vulkan [-1 … 1] range. When specifying a matrix for a geometry layer, this default matrix is overwritten.
When multiple views are specified, the layer is drawn multiple times using the specified layer views.
It’s possible to specify a negative term for height, which flips the image. When using a negative height, one should also adjust the y value to point to the lower left corner of the viewport instead of the upper left corner.
Attributes
offset_x, offset_y (float) Offset of top-left corner of the view. (0, 0) is the upper left and (1, 1) is the lower right. width (float) Normalized width (range [0.0, 1.0]). height (float) Normalized height (range [0.0, 1.0]). matrix (sequence of float) 16-elements representing a 4x4 transformation matrix. - property height
- property matrix
- property offset_x
- property offset_y
- property width
- __init__(*args, **kwargs)
Overloaded function.
__init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec, arg0: str, arg1: holoscan.operators.holoviz._holoviz.HolovizOp.InputType) -> None
InputSpec for the HolovizOp operator.
- Parameters
- tensor_name
- type
The tensor name for this input.
The type of data that this tensor represents.
Attributes
type (holoscan.operators.HolovizOp.InputType) The type of data that this tensor represents. opacity (float) The opacity of the object. Must be in range [0.0, 1.0] where 1.0 is fully opaque. priority (int) Layer priority, determines the render order. Layers with higher priority values are rendered on top of layers with lower priority. color (4-tuple of float) RGBA values in range [0.0, 1.0] for rendered geometry. line_width (float) Line width for geometry made of lines. point_size (float) Point size for geometry made of points. text (sequence of str) Sequence of strings used when type is HolovizOp.InputType.TEXT. depth_map_render_mode (holoscan.operators.HolovizOp.DepthMapRenderMode) The depth map render mode. Used only if type is HolovizOp.InputType.DEPTH_MAP or HolovizOp.InputType.DEPTH_MAP_COLOR. views (list of HolovizOp.InputSpec.View) Sequence of layer views. By default a layer will fill the whole window. When using a view, the layer can be placed freely within the window. When multiple views are specified, the layer is drawn multiple times using the specified layer views. 2. __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec, arg0: str, arg1: str) -> None
- property color
- property depth_map_render_mode
- description(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputSpec) → str
- Returns
- description
YAML string representation of the InputSpec class.
- property line_width
- property opacity
- property point_size
- property priority
- property text
- property type
- property views
- class InputType
Bases:
pybind11_builtins.pybind11_object
Members:
UNKNOWN
COLOR
COLOR_LUT
POINTS
LINES
LINE_STRIP
TRIANGLES
CROSSES
RECTANGLES
OVALS
TEXT
DEPTH_MAP
DEPTH_MAP_COLOR
POINTS_3D
LINES_3D
LINE_STRIP_3D
TRIANGLES_3D
Attributes
<a href="#holoscan.operators.HolovizOp.InputType.name">name</a>
value - COLOR = <InputType.COLOR: 1>
- COLOR_LUT = <InputType.COLOR_LUT: 2>
- CROSSES = <InputType.CROSSES: 7>
- DEPTH_MAP = <InputType.DEPTH_MAP: 11>
- DEPTH_MAP_COLOR = <InputType.DEPTH_MAP_COLOR: 12>
- LINES = <InputType.LINES: 4>
- LINES_3D = <InputType.LINES_3D: 14>
- LINE_STRIP = <InputType.LINE_STRIP: 5>
- LINE_STRIP_3D = <InputType.LINE_STRIP_3D: 15>
- OVALS = <InputType.OVALS: 9>
- POINTS = <InputType.POINTS: 3>
- POINTS_3D = <InputType.POINTS_3D: 13>
- RECTANGLES = <InputType.RECTANGLES: 8>
- TEXT = <InputType.TEXT: 10>
- TRIANGLES = <InputType.TRIANGLES: 6>
- TRIANGLES_3D = <InputType.TRIANGLES_3D: 16>
- UNKNOWN = <InputType.UNKNOWN: 0>
- __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp.InputType, value: int) → None
- property name
- property value
- class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.HolovizOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.holoviz._holoviz.HolovizOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, receivers: List[holoscan.core._core.IOSpec] = [], tensors: List[holoscan::ops::HolovizOp::InputSpec] = [], color_lut: List[List[float]] = [], window_title: str = 'Holoviz', display_name: str = 'DP-0', width: int = 1920, height: int = 1080, framerate: int = 60, use_exclusive_display: bool = False, fullscreen: bool = False, headless: bool = False, enable_render_buffer_input: bool = False, enable_render_buffer_output: bool = False, enable_camera_pose_output: bool = False, font_path: str = '', cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'holoviz_op') → None
Holoviz visualization operator using Holoviz module.
This is a Vulkan-based visualizer.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor and/or nvidia::gxf::VideoBuffer
- input_specslist[holoscan.operators.HolovizOp.InputSpec] (optional)
- render_buffer_inputnvidia::gxf::VideoBuffer (optional)
Any number of upstream ports may be connected to this
receivers
port. This port can accept either VideoBuffers or Tensors. These inputs can be in either host or device memory. Each tensor or video buffer will result in a layer. The operator autodetects the layer type for certain input types (e.g. a video buffer will result in an image layer). For other input types or more complex use cases, input specifications can be provided either at initialization time as a parameter or dynamically at run time (viainput_specs
). On each call tocompute
, tensors corresponding to all names specified in thetensors
parameter must be found or an exception will be raised. Any extra, named tensors not present in thetensors
parameter specification (or optional, dynamicinput_specs
input) will be ignored.A list of
InputSpec
objects. This port can be used to dynamically update the overlay specification at run time. No inputs are required on this port in order for the operator tocompute
.An empty render buffer can optionally be provided. The video buffer must have format GXF_VIDEO_FORMAT_RGBA and be in device memory. This input port only exists if
enable_render_buffer_input
was set toTrue
, in which casecompute
will only be called when a message arrives on this input.==Named Outputs==
- render_buffer_outputnvidia::gxf::VideoBuffer (optional)
- camera_pose_outputstd::array<float, 16> (optional)
Output for a filled render buffer. If an input render buffer is specified, it is using that one, else it allocates a new buffer. The video buffer will have format GXF_VIDEO_FORMAT_RGBA and will be in device memory. This output is useful for offline rendering or headless mode. This output port only exists if
enable_render_buffer_output
was set toTrue
.The camera pose. The parameters returned represent the values of a 4x4 row major projection matrix. This output port only exists if
enable_camera_pose_output
was set toTrue
.- Parameters
- fragment
- allocator
- receivers
- tensors
- color_lut
- window_title
- display_name
- width
- height
- framerate
- use_exclusive_display
- fullscreen
- headless
- enable_render_buffer_input
- enable_render_buffer_output
- enable_camera_pose_output
- font_path
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Allocator used to allocate render buffer output. If
None
, will default toholoscan.core.UnboundedAllocator
.List of input receivers.
List of input tensors. Each tensor is defined by a dictionary where the
"name"
key must correspond to a tensor sent to the operator’s input. See the notes section below for further details on how the tensor dictionary is defined.Color lookup table for tensors of type
color_lut
. Should be shape(n_colors, 4)
.Title on window canvas. Default value is
"Holoviz"
.In exclusive mode, name of display to use as shown with xrandr. Default value is
"DP-0"
.Window width or display resolution width if in exclusive or fullscreen mode. Default value is
1920
.Window height or display resolution width if in exclusive or fullscreen mode. Default value is
1080
.Display framerate in Hz if in exclusive mode. Default value is
60.0
.Enable exclusive display. Default value is
False
.Enable fullscreen window. Default value is
False
.Enable headless mode. No window is opened, the render buffer is output to port
render_buffer_output
. Default value isFalse
.If
True
, an additional input port, named"render_buffer_input"
is added to the operator. Default value isFalse
.If
True
, an additional output port, named"render_buffer_output"
is added to the operator. Default value isFalse
.If
True
, an additional output port, named"camera_pose_output"
is added to the operator. Default value isFalse
.File path for the font used for rendering text. Default value is
""
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"holoviz_op"
.
Notes
The
tensors
argument is used to specify the tensors to display. Each tensor is defined using a dictionary, that must, at minimum include a ‘name’ key that corresponds to a tensor found on the operator’s input. A ‘type’ key should also be provided to indicate the type of entry to display. The ‘type’ key will be one of {"color"
,"color_lut"
,"crosses"
,"lines"
,"lines_3d"
,"line_strip"
,"line_strip_3d"
,"ovals"
,"points"
,"points_3d"
,"rectangles"
,"text"
,"triangles"
,"triangles_3d"
,"depth_map"
,"depth_map_color"
,"unknown"
}. The default type is"unknown"
which will attempt to guess the corresponding type based on the tensor dimensions. Concrete examples are given below.To show a single 2D RGB or RGBA image, use a list containing a single tensor of type
"color"
.tensors = [dict(name="video", type="color", opacity=1.0, priority=0)]
Here, the optional key
opacity
is used to scale the opacity of the tensor. Thepriority
key is used to specify the render priority for layers. Layers with a higher priority will be rendered on top of those with a lower priority.If we also had a
"boxes"`
tensor representing rectangular bounding boxes, we could display them on top of the image like this.tensors = [ dict(name="video", type="color", priority=0), dict(name="boxes", type="rectangles", color=[1.0, 0.0, 0.0], line_width=2, priority=1), ]
where the
color
andline_width
keys specify the color and line width of the bounding box.The details of the dictionary is as follows:
name: name of the tensor containing the input data to display
type:
str
type: input type (default
"unknown"
)type:
str
possible values:
unknown: unknown type, the operator tries to guess the type by inspecting the tensor.
color: RGB or RGBA color 2d image.
color_lut: single channel 2d image, color is looked up.
points: point primitives, one coordinate (x, y) per primitive.
lines: line primitives, two coordinates (x0, y0) and (x1, y1) per primitive.
line_strip: line strip primitive, a line primitive i is defined by each coordinate (xi, yi) and the following (xi+1, yi+1).
triangles: triangle primitive, three coordinates (x0, y0), (x1, y1) and (x2, y2) per primitive.
crosses: cross primitive, a cross is defined by the center coordinate and the size (xi, yi, si).
rectangles: axis aligned rectangle primitive, each rectangle is defined by two coordinates (xi, yi) and (xi+1, yi+1).
ovals: oval primitive, an oval primitive is defined by the center coordinate and the axis sizes (xi, yi, sxi, syi).
text: text is defined by the top left coordinate and the size (x, y, s) per string, text strings are defined by InputSpec member text.
depth_map: single channel 2d array where each element represents a depth value. The data is rendered as a 3d object using points, lines or triangles. The color for the elements can be specified through
depth_map_color
. Supported format: 8-bit unsigned normalized format that has a single 8-bit depth component.depth_map_color: RGBA 2d image, same size as the depth map. One color value for each element of the depth map grid. Supported format: 32-bit unsigned normalized format that has an 8-bit R component in byte 0, an 8-bit G component in byte 1, an 8-bit B component in byte 2, and an 8-bit A component in byte 3.
opacity: layer opacity, 1.0 is fully opaque, 0.0 is fully transparent (default:
1.0
)type:
float
- priority: layer priority, determines the render order, layers with higher priority
values are rendered on top of layers with lower priority values (default:
0
)type:
int
color: RGBA color of rendered geometry (default:
[1.f, 1.f, 1.f, 1.f]
)type:
List[float]
line_width: line width for geometry made of lines (default:
1.0
)type:
float
point_size: point size for geometry made of points (default:
1.0
)type:
float
text: array of text strings, used when
type
is text (default:[]
)type:
List[str]
depth_map_render_mode: depth map render mode (default:
points
)type:
str
possible values:
points: render as points
lines: render as lines
triangles: render as triangles
Displaying Color Images
Image data can either be on host or device (GPU). Multiple image formats are supported
R 8 bit unsigned
R 16 bit unsigned
R 16 bit float
R 32 bit unsigned
R 32 bit float
RGB 8 bit unsigned
BGR 8 bit unsigned
RGBA 8 bit unsigned
BGRA 8 bit unsigned
RGBA 16 bit unsigned
RGBA 16 bit float
RGBA 32 bit float
When the
type
parameter is set tocolor_lut
the final color is looked up using the values from thecolor_lut
parameter. For color lookups these image formats are supportedR 8 bit unsigned
R 16 bit unsigned
R 32 bit unsigned
Drawing Geometry
In all cases,
x
andy
are normalized coordinates in the range[0, 1]
. Thex
andy
correspond to the horizontal and vertical axes of the display, respectively. The origin(0, 0)
is at the top left of the display. Geometric primitives outside of the visible area are clipped. Coordinate arrays are expected to have the shape(N, C)
whereN
is the coordinate count andC
is the component count for each coordinate.Points are defined by a
(x, y)
coordinate pair.Lines are defined by a set of two
(x, y)
coordinate pairs.Lines strips are defined by a sequence of
(x, y)
coordinate pairs. The first two coordinates define the first line, each additional coordinate adds a line connecting to the previous coordinate.Triangles are defined by a set of three
(x, y)
coordinate pairs.Crosses are defined by
(x, y, size)
tuples.size
specifies the size of the cross in thex
direction and is optional, if omitted it’s set to0.05
. The size in they
direction is calculated using the aspect ratio of the window to make the crosses square.Rectangles (bounding boxes) are defined by a pair of 2-tuples defining the upper-left and lower-right coordinates of a box:
(x1, y1), (x2, y2)
.Ovals are defined by
(x, y, size_x, size_y)
tuples.size_x
andsize_y
are optional, if omitted they are set to0.05
.Texts are defined by
(x, y, size)
tuples.size
specifies the size of the text iny
direction and is optional, if omitted it’s set to0.05
. The size in thex
direction is calculated using the aspect ratio of the window. The index of each coordinate references a text string from thetext
parameter and the index is clamped to the size of the text array. For example, if there is one item set for thetext
parameter, e.g.text=["my_text"]
and three coordinates, thenmy_text
is rendered three times. Iftext=["first text", "second text"]
and three coordinates are specified, thenfirst text
is rendered at the first coordinate,second text
at the second coordinate and thensecond text
again at the third coordinate. Thetext
string array is fixed and can’t be changed after initialization. To hide text which should not be displayed, specify coordinates greater than(1.0, 1.0)
for the text item, the text is then clipped away.3D Points are defined by a
(x, y, z)
coordinate tuple.3D Lines are defined by a set of two
(x, y, z)
coordinate tuples.3D Lines strips are defined by a sequence of
(x, y, z)
coordinate tuples. The first two coordinates define the first line, each additional coordinate adds a line connecting to the previous coordinate.3D Triangles are defined by a set of three
(x, y, z)
coordinate tuples.
Displaying Depth Maps
When
type
isdepth_map
the provided data is interpreted as a rectangular array of depth values. Additionally a 2d array with a color value for each point in the grid can be specified by settingtype
todepth_map_color
.The type of geometry drawn can be selected by setting
depth_map_render_mode
.Depth maps are rendered in 3D and support camera movement. The camera is controlled using the mouse:
Orbit (LMB)
Pan (LMB + CTRL | MMB)
Dolly (LMB + SHIFT | RMB | Mouse wheel)
Look Around (LMB + ALT | LMB + CTRL + SHIFT)
Zoom (Mouse wheel + SHIFT)
Output
By default a window is opened to display the rendering, but the extension can also be run in headless mode with the
headless
parameter.Using a display in exclusive mode is also supported with the
use_exclusive_display
parameter. This reduces the latency by avoiding the desktop compositor.The rendered framebuffer can be output to
render_buffer_output
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.holoviz._holoviz.HolovizOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.holoviz._holoviz.HolovizOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.InferenceOp
Bases:
holoscan.core._core.Operator
Inference operator.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor(s)
Any number of upstream ports may be connected to this
receivers
port. The operator will search across all messages for tensors matching those specified inin_tensor_names
. These are the set of input tensors used by the models ininference_map
.==Named Outputs==
- transmitternvidia::gxf::Tensor(s)
A message containing tensors corresponding to the inference results from all models will be emitted. The names of the tensors transmitted correspond to those in
out_tensor_names
.For more details on
InferenceOp
parameters, see [Customizing the Inference Operator](https://docs.nvidia.com/holoscan/sdk-user-guide/examples/byom.html#customizing-the-inference-operator) or refer to [Inference](https://docs.nvidia.com/holoscan/sdk-user-guide/inference.html).- Parameters
- fragment
- backend
- allocator
- inference_map
- model_path_map
- pre_processor_map
- device_map
- backend_map
- in_tensor_names
- out_tensor_names
- infer_on_cpu
- parallel_inference
- input_on_cuda
- output_on_cuda
- transmit_on_cuda
- enable_fp16
- is_engine_path
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Backend to use for inference. Set
"trt"
for TensorRT,"torch"
for LibTorch and"onnxrt"
for the ONNX runtime.Memory allocator to use for the output.
Tensor to model map.
Path to the ONNX model to be loaded.
Pre processed data to model map.
Mapping of model to GPU ID for inference.
Mapping of model to backend type for inference. Backend options:
"trt"
or"torch"
Input tensors.
Output tensors.
Whether to run the computation on the CPU instead of GPU. Default value is
False
.Whether to enable parallel execution. Default value is
True
.Whether the input buffer is on the GPU. Default value is
True
.Whether the output buffer is on the GPU. Default value is
True
.Whether to transmit the message on the GPU. Default value is
True
.Use 16-bit floating point computations. Default value is
False
.Whether the input model path mapping is for trt engine files. Default value is
False
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"inference"
.
Attributes
<a href="#holoscan.operators.InferenceOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.InferenceOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.InferenceOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.InferenceOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.InferenceOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.InferenceOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.InferenceOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.InferenceOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.InferenceOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.InferenceOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.InferenceOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.InferenceOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.InferenceOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.InferenceOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.InferenceOp.stop">stop</a>
(self)Operator stop method. DataMap DataVecMap OperatorType - class DataMap
Bases:
pybind11_builtins.pybind11_object
Methods
<a href="#holoscan.operators.InferenceOp.DataMap.get_map">get_map</a>
(self)<a href="#holoscan.operators.InferenceOp.DataMap.insert">insert</a>
(self)- __init__(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → None
- get_map(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → Dict[str, str]
- insert(self: holoscan.operators.inference._inference.InferenceOp.DataMap) → Dict[str, str]
- class DataVecMap
Bases:
pybind11_builtins.pybind11_object
Methods
<a href="#holoscan.operators.InferenceOp.DataVecMap.get_map">get_map</a>
(self)<a href="#holoscan.operators.InferenceOp.DataVecMap.insert">insert</a>
(self)- __init__(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → None
- get_map(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → Dict[str, List[str]]
- insert(self: holoscan.operators.inference._inference.InferenceOp.DataVecMap) → Dict[str, List[str]]
- class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.InferenceOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.inference._inference.InferenceOp, fragment: holoscan.core._core.Fragment, backend: str, allocator: holoscan.resources._resources.Allocator, inference_map: dict, model_path_map: dict, pre_processor_map: dict, device_map: dict = {}, backend_map: dict = {}, in_tensor_names: List[str] = [], out_tensor_names: List[str] = [], infer_on_cpu: bool = False, parallel_inference: bool = True, input_on_cuda: bool = True, output_on_cuda: bool = True, transmit_on_cuda: bool = True, enable_fp16: bool = False, is_engine_path: bool = False, cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'inference') → None
Inference operator.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor(s)
Any number of upstream ports may be connected to this
receivers
port. The operator will search across all messages for tensors matching those specified inin_tensor_names
. These are the set of input tensors used by the models ininference_map
.==Named Outputs==
- transmitternvidia::gxf::Tensor(s)
A message containing tensors corresponding to the inference results from all models will be emitted. The names of the tensors transmitted correspond to those in
out_tensor_names
.For more details on
InferenceOp
parameters, see [Customizing the Inference Operator](https://docs.nvidia.com/holoscan/sdk-user-guide/examples/byom.html#customizing-the-inference-operator) or refer to [Inference](https://docs.nvidia.com/holoscan/sdk-user-guide/inference.html).- Parameters
- fragment
- backend
- allocator
- inference_map
- model_path_map
- pre_processor_map
- device_map
- backend_map
- in_tensor_names
- out_tensor_names
- infer_on_cpu
- parallel_inference
- input_on_cuda
- output_on_cuda
- transmit_on_cuda
- enable_fp16
- is_engine_path
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Backend to use for inference. Set
"trt"
for TensorRT,"torch"
for LibTorch and"onnxrt"
for the ONNX runtime.Memory allocator to use for the output.
Tensor to model map.
Path to the ONNX model to be loaded.
Pre processed data to model map.
Mapping of model to GPU ID for inference.
Mapping of model to backend type for inference. Backend options:
"trt"
or"torch"
Input tensors.
Output tensors.
Whether to run the computation on the CPU instead of GPU. Default value is
False
.Whether to enable parallel execution. Default value is
True
.Whether the input buffer is on the GPU. Default value is
True
.Whether the output buffer is on the GPU. Default value is
True
.Whether to transmit the message on the GPU. Default value is
True
.Use 16-bit floating point computations. Default value is
False
.Whether the input model path mapping is for trt engine files. Default value is
False
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"inference"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.inference._inference.InferenceOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.inference._inference.InferenceOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.InferenceProcessorOp
Bases:
holoscan.core._core.Operator
Holoinfer Processing operator.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor(s)
Any number of upstream ports may be connected to this
receivers
port. The operator will search across all messages for tensors matching those specified inin_tensor_names
. These are the set of input tensors used by the processing operations specified inprocess_map
.==Named Outputs==
- transmitternvidia::gxf::Tensor(s)
A message containing tensors corresponding to the processed results from operations will be emitted. The names of the tensors transmitted correspond to those in
out_tensor_names
.- Parameters
- fragment
- allocator
- process_operations
- processed_map
- in_tensor_names
- out_tensor_names
- input_on_cuda
- output_on_cuda
- transmit_on_cuda
- cuda_stream_pool
- config_path
- disable_transmitter
- name
The fragment that the operator belongs to.
Memory allocator to use for the output.
Operations in sequence on tensors.
Input-output tensor mapping.
Names of input tensors in the order to be fed into the operator.
Names of output tensors in the order to be fed into the operator.
Whether the input buffer is on the GPU. Default value is
False
.Whether the output buffer is on the GPU. Default value is
False
.Whether to transmit the message on the GPU. Default value is
False
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.File path to the config file. Default value is
""
.If
True
, disable the transmitter output port of the operator. Default value isFalse
.The name of the operator. Default value is
"postprocessor"
.
Attributes
<a href="#holoscan.operators.InferenceProcessorOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.InferenceProcessorOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.InferenceProcessorOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.InferenceProcessorOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.InferenceProcessorOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.InferenceProcessorOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.InferenceProcessorOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.InferenceProcessorOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.InferenceProcessorOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.InferenceProcessorOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.InferenceProcessorOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.InferenceProcessorOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.InferenceProcessorOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.InferenceProcessorOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.InferenceProcessorOp.stop">stop</a>
(self)Operator stop method. DataMap DataVecMap OperatorType - class DataMap
Bases:
pybind11_builtins.pybind11_object
Methods
<a href="#holoscan.operators.InferenceProcessorOp.DataMap.get_map">get_map</a>
(self)<a href="#holoscan.operators.InferenceProcessorOp.DataMap.insert">insert</a>
(self)- __init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → None
- get_map(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → Dict[str, str]
- insert(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataMap) → Dict[str, str]
- class DataVecMap
Bases:
pybind11_builtins.pybind11_object
Methods
<a href="#holoscan.operators.InferenceProcessorOp.DataVecMap.get_map">get_map</a>
(self)<a href="#holoscan.operators.InferenceProcessorOp.DataVecMap.insert">insert</a>
(self)- __init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → None
- get_map(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → Dict[str, List[str]]
- insert(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp.DataVecMap) → Dict[str, List[str]]
- class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.InferenceProcessorOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, process_operations: dict = {}, processed_map: dict = {}, in_tensor_names: List[str] = [], out_tensor_names: List[str] = [], input_on_cuda: bool = False, output_on_cuda: bool = False, transmit_on_cuda: bool = False, disable_transmitter: bool = False, cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, config_path: str = '', name: str = 'postprocessor') → None
Holoinfer Processing operator.
==Named Inputs==
- receiversmulti-receiver accepting nvidia::gxf::Tensor(s)
Any number of upstream ports may be connected to this
receivers
port. The operator will search across all messages for tensors matching those specified inin_tensor_names
. These are the set of input tensors used by the processing operations specified inprocess_map
.==Named Outputs==
- transmitternvidia::gxf::Tensor(s)
A message containing tensors corresponding to the processed results from operations will be emitted. The names of the tensors transmitted correspond to those in
out_tensor_names
.- Parameters
- fragment
- allocator
- process_operations
- processed_map
- in_tensor_names
- out_tensor_names
- input_on_cuda
- output_on_cuda
- transmit_on_cuda
- cuda_stream_pool
- config_path
- disable_transmitter
- name
The fragment that the operator belongs to.
Memory allocator to use for the output.
Operations in sequence on tensors.
Input-output tensor mapping.
Names of input tensors in the order to be fed into the operator.
Names of output tensors in the order to be fed into the operator.
Whether the input buffer is on the GPU. Default value is
False
.Whether the output buffer is on the GPU. Default value is
False
.Whether to transmit the message on the GPU. Default value is
False
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.File path to the config file. Default value is
""
.If
True
, disable the transmitter output port of the operator. Default value isFalse
.The name of the operator. Default value is
"postprocessor"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.inference_processor._inference_processor.InferenceProcessorOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.NTV2Channel
Bases:
pybind11_builtins.pybind11_object
Members:
NTV2_CHANNEL1
NTV2_CHANNEL2
NTV2_CHANNEL3
NTV2_CHANNEL4
NTV2_CHANNEL5
NTV2_CHANNEL6
NTV2_CHANNEL7
NTV2_CHANNEL8
NTV2_MAX_NUM_CHANNELS
NTV2_CHANNEL_INVALID
Attributes
<a href="#holoscan.operators.NTV2Channel.name">name</a>
value - NTV2_CHANNEL1 = <NTV2Channel.NTV2_CHANNEL1: 0>
- NTV2_CHANNEL2 = <NTV2Channel.NTV2_CHANNEL2: 1>
- NTV2_CHANNEL3 = <NTV2Channel.NTV2_CHANNEL3: 2>
- NTV2_CHANNEL4 = <NTV2Channel.NTV2_CHANNEL4: 3>
- NTV2_CHANNEL5 = <NTV2Channel.NTV2_CHANNEL5: 4>
- NTV2_CHANNEL6 = <NTV2Channel.NTV2_CHANNEL6: 5>
- NTV2_CHANNEL7 = <NTV2Channel.NTV2_CHANNEL7: 6>
- NTV2_CHANNEL8 = <NTV2Channel.NTV2_CHANNEL8: 7>
- NTV2_CHANNEL_INVALID = <NTV2Channel.NTV2_MAX_NUM_CHANNELS: 8>
- NTV2_MAX_NUM_CHANNELS = <NTV2Channel.NTV2_MAX_NUM_CHANNELS: 8>
- __init__(self: holoscan.operators.aja_source._aja_source.NTV2Channel, value: int) → None
- property name
- property value
- class holoscan.operators.PingRxOp(fragment, *args, **kwargs)
Bases:
<a href="holoscan_python_api_core.html#holoscan.core.Operator">holoscan.core.Operator</a>
Simple receiver operator.
This is an example of a native operator with one input port. On each tick, it receives an integer from the “in” port.
==Named Inputs==
- inany
A received value.
Attributes
<a href="#holoscan.operators.PingRxOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.PingRxOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.PingRxOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.PingRxOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.PingRxOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.PingRxOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.PingRxOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.PingRxOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.PingRxOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.PingRxOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.PingRxOp.compute">compute</a>
(op_input, op_output, context)Default implementation of compute <a href="#holoscan.operators.PingRxOp.initialize">initialize</a>
()Default implementation of initialize <a href="#holoscan.operators.PingRxOp.setup">setup</a>
(spec)Default implementation of setup method. <a href="#holoscan.operators.PingRxOp.start">start</a>
()Default implementation of start <a href="#holoscan.operators.PingRxOp.stop">stop</a>
()Default implementation of stop OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.PingRxOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.core._core.Operator, arg0: object, arg1: holoscan::Fragment, *args, **kwargs) → None
Operator class.
Can be initialized with any number of Python positional and keyword arguments.
If a name keyword argument is provided, it must be a str and will be used to set the name of the Operator.
Condition classes will be added to
self.conditions
, Resource classes will be added toself.resources
, and any other arguments will be cast from a Python argument type to a C++ Arg and stored inself.args
. (For details on how the casting is done, see the py_object_to_arg utility). When a Condition or Resource is provided via a kwarg, it’s name will be automatically be updated to the name of the kwarg.- Parameters
- fragment
- *args
- **kwargs
The holoscan.core.Fragment (or holoscan.core.Application) to which this Operator will belong.
Positional arguments.
Keyword arguments.
- Raises
- RuntimeError
If name kwarg is provided, but is not of str type. If multiple arguments of type Fragment are provided. If any other arguments cannot be converted to Arg type via py_object_to_arg.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(op_input, op_output, context)
Default implementation of compute
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize()
Default implementation of initialize
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(spec: holoscan.core._core.PyOperatorSpec)
Default implementation of setup method.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start()
Default implementation of start
- stop()
Default implementation of stop
- class holoscan.operators.PingTxOp(fragment, *args, **kwargs)
Bases:
<a href="holoscan_python_api_core.html#holoscan.core.Operator">holoscan.core.Operator</a>
Simple transmitter operator.
On each tick, it transmits an integer to the “out” port.
==Named Outputs==
- outint
An index value that increments by one on each call to compute. The starting value is 1.
Attributes
<a href="#holoscan.operators.PingTxOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.PingTxOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.PingTxOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.PingTxOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.PingTxOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.PingTxOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.PingTxOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.PingTxOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.PingTxOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.PingTxOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.PingTxOp.compute">compute</a>
(op_input, op_output, context)Default implementation of compute <a href="#holoscan.operators.PingTxOp.initialize">initialize</a>
()Default implementation of initialize <a href="#holoscan.operators.PingTxOp.setup">setup</a>
(spec)Default implementation of setup method. <a href="#holoscan.operators.PingTxOp.start">start</a>
()Default implementation of start <a href="#holoscan.operators.PingTxOp.stop">stop</a>
()Default implementation of stop OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.PingTxOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.core._core.Operator, arg0: object, arg1: holoscan::Fragment, *args, **kwargs) → None
Operator class.
Can be initialized with any number of Python positional and keyword arguments.
If a name keyword argument is provided, it must be a str and will be used to set the name of the Operator.
Condition classes will be added to
self.conditions
, Resource classes will be added toself.resources
, and any other arguments will be cast from a Python argument type to a C++ Arg and stored inself.args
. (For details on how the casting is done, see the py_object_to_arg utility). When a Condition or Resource is provided via a kwarg, it’s name will be automatically be updated to the name of the kwarg.- Parameters
- fragment
- *args
- **kwargs
The holoscan.core.Fragment (or holoscan.core.Application) to which this Operator will belong.
Positional arguments.
Keyword arguments.
- Raises
- RuntimeError
If name kwarg is provided, but is not of str type. If multiple arguments of type Fragment are provided. If any other arguments cannot be converted to Arg type via py_object_to_arg.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(op_input, op_output, context)
Default implementation of compute
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize()
Default implementation of initialize
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(spec: holoscan.core._core.PyOperatorSpec)
Default implementation of setup method.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start()
Default implementation of start
- stop()
Default implementation of stop
- class holoscan.operators.SegmentationPostprocessorOp
Bases:
holoscan.core._core.Operator
Operator carrying out post-processing operations on segmentation outputs.
==Named Inputs==
- in_tensornvidia::gxf::Tensor
Expects a message containing a 32-bit floating point tensor with name
in_tensor_name
. The expected data layout of this tensor is HWC, NCHW or NHWC format as specified viadata_format
.==Named Outputs==
- out_tensornvidia::gxf::Tensor
Emits a message containing a tensor named “out_tensor” that contains the segmentation labels. This tensor will have unsigned 8-bit integer data type and shape (H, W, 1).
- Parameters
- fragment
- allocator
- in_tensor_name
- network_output_type
- data_format
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Memory allocator to use for the output.
Name of the input tensor. Default value is
""
.Network output type (e.g. ‘softmax’). Default value is
"softmax"
.Data format of network output. Default value is
"hwc"
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"segmentation_postprocessor"
.
Attributes
<a href="#holoscan.operators.SegmentationPostprocessorOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.SegmentationPostprocessorOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.SegmentationPostprocessorOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.SegmentationPostprocessorOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.SegmentationPostprocessorOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.SegmentationPostprocessorOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.SegmentationPostprocessorOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.SegmentationPostprocessorOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.SegmentationPostprocessorOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.SegmentationPostprocessorOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.SegmentationPostprocessorOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.SegmentationPostprocessorOp.initialize">initialize</a>
(self)Operator initialization method. <a href="#holoscan.operators.SegmentationPostprocessorOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.SegmentationPostprocessorOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.SegmentationPostprocessorOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.SegmentationPostprocessorOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.segmentation_postprocessor._segmentation_postprocessor.SegmentationPostprocessorOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, in_tensor_name: str = '', network_output_type: str = 'softmax', data_format: str = 'hwc', cuda_stream_pool: holoscan.resources._resources.CudaStreamPool = None, name: str = 'segmentation_postprocessor') → None
Operator carrying out post-processing operations on segmentation outputs.
==Named Inputs==
- in_tensornvidia::gxf::Tensor
Expects a message containing a 32-bit floating point tensor with name
in_tensor_name
. The expected data layout of this tensor is HWC, NCHW or NHWC format as specified viadata_format
.==Named Outputs==
- out_tensornvidia::gxf::Tensor
Emits a message containing a tensor named “out_tensor” that contains the segmentation labels. This tensor will have unsigned 8-bit integer data type and shape (H, W, 1).
- Parameters
- fragment
- allocator
- in_tensor_name
- network_output_type
- data_format
- cuda_stream_pool
- name
The fragment that the operator belongs to.
Memory allocator to use for the output.
Name of the input tensor. Default value is
""
.Network output type (e.g. ‘softmax’). Default value is
"softmax"
.Data format of network output. Default value is
"hwc"
.holoscan.resources.CudaStreamPool
instance to allocate CUDA streams. Default value isNone
.The name of the operator. Default value is
"segmentation_postprocessor"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.core._core.Operator) → None
Operator initialization method.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.segmentation_postprocessor._segmentation_postprocessor.SegmentationPostprocessorOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.V4L2VideoCaptureOp
Bases:
holoscan.core._core.Operator
Operator to get a video stream from a V4L2 source.
https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/v4l2.html
Inputs a video stream from a V4L2 node, including USB cameras and HDMI IN.
Input stream is on host. If no pixel format is specified in the yaml configuration file, the pixel format will be automatically selected. However, only
AB24
andYUYV
are then supported. If a pixel format is specified in the yaml file, then this format will be used. However, note that the operator then expects that this format can be encoded as RGBA32. If not, the behavior is undefined.Output stream is on host. Always RGBA32 at this time.
Use
holoscan.operators.FormatConverterOp
to move data from the host to a GPU device.==Named Outputs==
- signalnvidia::gxf::VideoBuffer
A message containing a video buffer on the host with format GXF_VIDEO_FORMAT_RGBA.
- Parameters
- fragment
- allocator
- device
- width
- height
- num_buffers
- pixel_format
- name
- exposure_time
- gain
The fragment that the operator belongs to.
Memory allocator to use for the output.
The device to target (e.g. “/dev/video0” for device 0). Default value is
"/dev/video0"
.Width of the video stream. Default value is
0
.Height of the video stream. Default value is
0
.Number of V4L2 buffers to use. Default value is
4
.Video stream pixel format (little endian four character code (fourcc)). Default value is
"auto"
.The name of the operator. Default value is
"v4l2_video_capture"
.Exposure time of the camera sensor in multiples of 100 μs (e.g. setting exposure_time to 100 is 10 ms). Default: auto exposure, or camera sensor default. Use v4l2-ctl -d /dev/<your_device> -L for a range of values supported by your device. When not set by the user, V4L2_CID_EXPOSURE_AUTO is set to V4L2_EXPOSURE_AUTO, or to V4L2_EXPOSURE_APERTURE_PRIORITY if the former is not supported. When set by the user, V4L2_CID_EXPOSURE_AUTO is set to V4L2_EXPOSURE_SHUTTER_PRIORITY, or to V4L2_EXPOSURE_MANUAL if the former is not supported. The provided value is then used to set V4L2_CID_EXPOSURE_ABSOLUTE.
Gain of the camera sensor. Default: auto gain, or camera sensor default. Use v4l2-ctl -d /dev/<your_device> -L for a range of values supported by your device. When not set by the user, V4L2_CID_AUTOGAIN is set to true (if supported). When set by the user, V4L2_CID_AUTOGAIN is set to false (if supported). The provided value is then used to set V4L2_CID_GAIN.
Attributes
<a href="#holoscan.operators.V4L2VideoCaptureOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.V4L2VideoCaptureOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.V4L2VideoCaptureOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.V4L2VideoCaptureOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.V4L2VideoCaptureOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.V4L2VideoCaptureOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.V4L2VideoCaptureOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.V4L2VideoCaptureOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.V4L2VideoCaptureOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.V4L2VideoCaptureOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.V4L2VideoCaptureOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.V4L2VideoCaptureOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.V4L2VideoCaptureOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.V4L2VideoCaptureOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.V4L2VideoCaptureOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.V4L2VideoCaptureOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp, fragment: holoscan.core._core.Fragment, allocator: holoscan.resources._resources.Allocator, device: str = '0', width: int = 0, height: int = 0, num_buffers: int = 4, pixel_format: str = 'auto', name: str = 'v4l2_video_capture', exposure_time: Optional[int] = None, gain: Optional[int] = None) → None
Operator to get a video stream from a V4L2 source.
https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/v4l2.html
Inputs a video stream from a V4L2 node, including USB cameras and HDMI IN.
Input stream is on host. If no pixel format is specified in the yaml configuration file, the pixel format will be automatically selected. However, only
AB24
andYUYV
are then supported. If a pixel format is specified in the yaml file, then this format will be used. However, note that the operator then expects that this format can be encoded as RGBA32. If not, the behavior is undefined.Output stream is on host. Always RGBA32 at this time.
Use
holoscan.operators.FormatConverterOp
to move data from the host to a GPU device.==Named Outputs==
- signalnvidia::gxf::VideoBuffer
A message containing a video buffer on the host with format GXF_VIDEO_FORMAT_RGBA.
- Parameters
- fragment
- allocator
- device
- width
- height
- num_buffers
- pixel_format
- name
- exposure_time
- gain
The fragment that the operator belongs to.
Memory allocator to use for the output.
The device to target (e.g. “/dev/video0” for device 0). Default value is
"/dev/video0"
.Width of the video stream. Default value is
0
.Height of the video stream. Default value is
0
.Number of V4L2 buffers to use. Default value is
4
.Video stream pixel format (little endian four character code (fourcc)). Default value is
"auto"
.The name of the operator. Default value is
"v4l2_video_capture"
.Exposure time of the camera sensor in multiples of 100 μs (e.g. setting exposure_time to 100 is 10 ms). Default: auto exposure, or camera sensor default. Use v4l2-ctl -d /dev/<your_device> -L for a range of values supported by your device. When not set by the user, V4L2_CID_EXPOSURE_AUTO is set to V4L2_EXPOSURE_AUTO, or to V4L2_EXPOSURE_APERTURE_PRIORITY if the former is not supported. When set by the user, V4L2_CID_EXPOSURE_AUTO is set to V4L2_EXPOSURE_SHUTTER_PRIORITY, or to V4L2_EXPOSURE_MANUAL if the former is not supported. The provided value is then used to set V4L2_CID_EXPOSURE_ABSOLUTE.
Gain of the camera sensor. Default: auto gain, or camera sensor default. Use v4l2-ctl -d /dev/<your_device> -L for a range of values supported by your device. When not set by the user, V4L2_CID_AUTOGAIN is set to true (if supported). When set by the user, V4L2_CID_AUTOGAIN is set to false (if supported). The provided value is then used to set V4L2_CID_GAIN.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.v4l2_video_capture._v4l2_video_capture.V4L2VideoCaptureOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
-
spec :
holoscan.core.OperatorSpec
The operator specification.
-
spec :
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.VideoStreamRecorderOp
Bases:
holoscan.core._core.Operator
Operator class to record a video stream to a file.
==Named Inputs==
- inputnvidia::gxf::Tensor
A message containing a video frame to serialize to disk. The input tensor can be on either the CPU or GPU. This data location will be recorded as part of the metadata serialized to disk and if the data is later read back in via VideoStreamReplayerOp, the tensor output of that operator will be on the same device (CPU or GPU).
- Parameters
- fragment
- directory
- basename
- flush_on_tick
- name
The fragment that the operator belongs to.
Directory path for storing files.
User specified file name without extension.
Flushes output buffer on every tick when
True
. Default value isFalse
.The name of the operator. Default value is
"video_stream_recorder"
.
Attributes
<a href="#holoscan.operators.VideoStreamRecorderOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.VideoStreamRecorderOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.VideoStreamRecorderOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.VideoStreamRecorderOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.VideoStreamRecorderOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.VideoStreamRecorderOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.VideoStreamRecorderOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.VideoStreamRecorderOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.VideoStreamRecorderOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.VideoStreamRecorderOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.VideoStreamRecorderOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.VideoStreamRecorderOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.VideoStreamRecorderOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.VideoStreamRecorderOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.VideoStreamRecorderOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.VideoStreamRecorderOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp, fragment: holoscan.core._core.Fragment, directory: str, basename: str, flush_on_tick: bool = False, name: str = 'recorder') → None
Operator class to record a video stream to a file.
==Named Inputs==
- inputnvidia::gxf::Tensor
A message containing a video frame to serialize to disk. The input tensor can be on either the CPU or GPU. This data location will be recorded as part of the metadata serialized to disk and if the data is later read back in via VideoStreamReplayerOp, the tensor output of that operator will be on the same device (CPU or GPU).
- Parameters
- fragment
- directory
- basename
- flush_on_tick
- name
The fragment that the operator belongs to.
Directory path for storing files.
User specified file name without extension.
Flushes output buffer on every tick when
True
. Default value isFalse
.The name of the operator. Default value is
"video_stream_recorder"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.video_stream_recorder._video_stream_recorder.VideoStreamRecorderOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.
- class holoscan.operators.VideoStreamReplayerOp
Bases:
holoscan.core._core.Operator
Operator class to replay a video stream from a file.
==Named Outputs==
- outputnvidia::gxf::Tensor
A message containing a video frame deserialized from disk. Depending on the metadata in the file being read, this tensor could be on either CPU or GPU. For the data used in examples distributed with the SDK, the tensor will be an unnamed GPU tensor (name == “”).
- Parameters
- fragment
- directory
- basename
- batch_size
- ignore_corrupted_entities
- frame_rate
- realtime
- repeat
- count
- name
The fragment that the operator belongs to.
Directory path for reading files from.
User specified file name without extension.
Number of entities to read and publish for one tick. Default value is
1
.If an entity could not be deserialized, it is ignored by default; otherwise a failure is generated. Default value is
True
.Frame rate to replay. If zero value is specified, it follows timings in timestamps. Default value is
0.0
.Playback video in realtime, based on frame_rate or timestamps. Default value is
True
.Repeat video stream in a loop. Default value is
False
.Number of frame counts to playback. If zero value is specified, it is ignored. If the count is less than the number of frames in the video, it would finish early. Default value is
0
.The name of the operator. Default value is
"video_stream_replayer"
.
Attributes
<a href="#holoscan.operators.VideoStreamReplayerOp.args">args</a>
The list of arguments associated with the component. <a href="#holoscan.operators.VideoStreamReplayerOp.conditions">conditions</a>
Conditions associated with the operator. <a href="#holoscan.operators.VideoStreamReplayerOp.description">description</a>
YAML formatted string describing the operator. <a href="#holoscan.operators.VideoStreamReplayerOp.fragment">fragment</a>
The fragment ( holoscan.core.Fragment
) that the operator belongs to.<a href="#holoscan.operators.VideoStreamReplayerOp.id">id</a>
The identifier of the component. <a href="#holoscan.operators.VideoStreamReplayerOp.name">name</a>
The name of the operator. <a href="#holoscan.operators.VideoStreamReplayerOp.operator_type">operator_type</a>
The operator type. <a href="#holoscan.operators.VideoStreamReplayerOp.resources">resources</a>
Resources associated with the operator. <a href="#holoscan.operators.VideoStreamReplayerOp.spec">spec</a>
The operator spec ( holoscan.core.OperatorSpec
) associated with the operator.Methods
<a href="#holoscan.operators.VideoStreamReplayerOp.add_arg">add_arg</a>
(*args, **kwargs)Overloaded function. <a href="#holoscan.operators.VideoStreamReplayerOp.compute">compute</a>
(self, arg0, arg1, arg2)Operator compute method. <a href="#holoscan.operators.VideoStreamReplayerOp.initialize">initialize</a>
(self)Initialize the operator. <a href="#holoscan.operators.VideoStreamReplayerOp.setup">setup</a>
(self, spec)Define the operator specification. <a href="#holoscan.operators.VideoStreamReplayerOp.start">start</a>
(self)Operator start method. <a href="#holoscan.operators.VideoStreamReplayerOp.stop">stop</a>
(self)Operator stop method. OperatorType - class OperatorType
Bases:
pybind11_builtins.pybind11_object
Enum class for operator types used by the executor.
NATIVE: Native operator.
GXF: GXF operator.
VIRTUAL: Virtual operator. (for internal use, not intended for use by application authors)
Members:
NATIVE
GXF
VIRTUAL
Attributes
<a href="#holoscan.operators.VideoStreamReplayerOp.OperatorType.name">name</a>
value - GXF = <OperatorType.GXF: 1>
- NATIVE = <OperatorType.NATIVE: 0>
- VIRTUAL = <OperatorType.VIRTUAL: 2>
- __init__(self: holoscan.core._core.Operator.OperatorType, value: int) → None
- property name
- property value
- __init__(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp, fragment: holoscan.core._core.Fragment, directory: str, basename: str, batch_size: int = 1, ignore_corrupted_entities: bool = True, frame_rate: float = 1.0, realtime: bool = True, repeat: bool = False, count: int = 0, name: str = 'format_converter') → None
Operator class to replay a video stream from a file.
==Named Outputs==
- outputnvidia::gxf::Tensor
A message containing a video frame deserialized from disk. Depending on the metadata in the file being read, this tensor could be on either CPU or GPU. For the data used in examples distributed with the SDK, the tensor will be an unnamed GPU tensor (name == “”).
- Parameters
- fragment
- directory
- basename
- batch_size
- ignore_corrupted_entities
- frame_rate
- realtime
- repeat
- count
- name
The fragment that the operator belongs to.
Directory path for reading files from.
User specified file name without extension.
Number of entities to read and publish for one tick. Default value is
1
.If an entity could not be deserialized, it is ignored by default; otherwise a failure is generated. Default value is
True
.Frame rate to replay. If zero value is specified, it follows timings in timestamps. Default value is
0.0
.Playback video in realtime, based on frame_rate or timestamps. Default value is
True
.Repeat video stream in a loop. Default value is
False
.Number of frame counts to playback. If zero value is specified, it is ignored. If the count is less than the number of frames in the video, it would finish early. Default value is
0
.The name of the operator. Default value is
"video_stream_replayer"
.
- add_arg(*args, **kwargs)
Overloaded function.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Arg) -> None
Add an argument to the component.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.ArgList) -> None
Add a list of arguments to the component.
add_arg(self: holoscan.core._core.Operator, **kwargs) -> None
Add arguments to the component via Python kwargs.
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Condition) -> None
add_arg(self: holoscan.core._core.Operator, arg: holoscan.core._core.Resource) -> None
Add a condition or resource to the Operator.
This can be used to add a condition or resource to an operator after it has already been constructed.
- Parameters
- arg
The condition or resource to add.
- property args
The list of arguments associated with the component.
- Returns
- arglist
- compute(self: holoscan.core._core.Operator, arg0: holoscan.core._core.InputContext, arg1: holoscan.core._core.OutputContext, arg2: holoscan.core._core.ExecutionContext) → None
Operator compute method. This method defines the primary computation to be executed by the operator.
- property conditions
Conditions associated with the operator.
- property description
YAML formatted string describing the operator.
- property fragment
The fragment (
holoscan.core.Fragment
) that the operator belongs to.- property id
The identifier of the component.
The identifier is initially set to
-1
, and will become a valid value when the component is initialized.With the default executor (holoscan.gxf.GXFExecutor), the identifier is set to the GXF component ID.
- Returns
- id
- initialize(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp) → None
Initialize the operator.
This method is called only once when the operator is created for the first time, and uses a light-weight initialization.
- property name
The name of the operator.
- property operator_type
The operator type.
holoscan.core.Operator.OperatorType enum representing the type of the operator. The two types currently implemented are native and GXF.
- property resources
Resources associated with the operator.
- setup(self: holoscan.operators.video_stream_replayer._video_stream_replayer.VideoStreamReplayerOp, spec: holoscan.core._core.OperatorSpec) → None
Define the operator specification.
- Parameters
- spec
The operator specification.
- property spec
The operator spec (
holoscan.core.OperatorSpec
) associated with the operator.- start(self: holoscan.core._core.Operator) → None
Operator start method.
- stop(self: holoscan.core._core.Operator) → None
Operator stop method.