Defined in File holoviz.hpp
Nested Types
Base Type
public holoscan::Operator
(Class Operator)
-
class HolovizOp : public holoscan::Operator
Operator class for data visualization.
This high-speed viewer handles compositing, blending, and visualization of RGB or RGBA images, masks, geometric primitives, text and depth maps. The operator can auto detect the format of the input tensors when only the
receivers
parameter list is specified.Displaying Color Images
Image data can either be on host or device (GPU). Multiple image formats are supported
R 8 bit unsigned
R 16 bit unsigned
R 16 bit float
R 32 bit unsigned
R 32 bit float
RGB 8 bit unsigned
BGR 8 bit unsigned
RGBA 8 bit unsigned
BGRA 8 bit unsigned
RGBA 16 bit unsigned
RGBA 16 bit float
RGBA 32 bit float
When the
type
parameter is set tocolor_lut
the final color is looked up using the values from thecolor_lut
parameter. For color lookups these image formats are supportedR 8 bit unsigned
R 16 bit unsigned
R 32 bit unsigned
Drawing Geometry
In all cases,
x
andy
are normalized coordinates in the range[0, 1]
. Thex
andy
correspond to the horizontal and vertical axes of the display, respectively. The origin(0, 0)
is at the top left of the display. All coordinates should be defined using a single precision float data type. Geometric primitives outside of the visible area are clipped. Coordinate arrays are expected to have the shape(1, N, C)
whereN
is the coordinate count andC
is the component count for each coordinate.Points are defined by a
(x, y)
coordinate pair.Lines are defined by a set of two
(x, y)
coordinate pairs.Lines strips are defined by a sequence of
(x, y)
coordinate pairs. The first two coordinates define the first line, each additional coordinate adds a line connecting to the previous coordinate.Triangles are defined by a set of three
(x, y)
coordinate pairs.Crosses are defined by
(x, y, size)
tuples.size
specifies the size of the cross in thex
direction and is optional, if omitted it’s set to0.05
. The size in they
direction is calculated using the aspect ratio of the window to make the crosses square.Rectangles (bounding boxes) are defined by a pair of 2-tuples defining the upper-left and lower-right coordinates of a box:
(x1, y1), (x2, y2)
.Ovals are defined by
(x, y, size_x, size_y)
tuples.size_x
andsize_y
are optional, if omitted they are set to0.05
.Texts are defined by
(x, y, size)
tuples.size
specifies the size of the text iny
direction and is optional, if omitted it’s set to0.05
. The size in thex
direction is calculated using the aspect ratio of the window. The index of each coordinate references a text string from thetext
parameter and the index is clamped to the size of the text array. For example, if there is one item set for thetext
parameter, e.g. text=[‘my_text]and three coordinates, then
my_textis rendered three times. If
text=[‘first text’, ‘second text’]and three coordinates are specified, then
first textis rendered at the first coordinate,
second textat the second coordinate and then
second textagain at the third coordinate. The
text` string array is fixed and can’t be changed after initialization. To hide text which should not be displayed, specify coordinates greater than(1.0, 1.0)
for the text item, the text is then clipped away.
Displaying Depth Maps
When
type
isdepth_map
the provided data is interpreted as a rectangular array of depth values. Additionally a 2d array with a color value for each point in the grid can be specified by settingtype
todepth_map_color
.The type of geometry drawn can be selected by setting
depth_map_render_mode
.Depth maps are rendered in 3D and support camera movement. The camera is controlled using the mouse:
Orbit (LMB)
Pan (LMB + CTRL | MMB)
Dolly (LMB + SHIFT | RMB | Mouse wheel)
Look Around (LMB + ALT | LMB + CTRL + SHIFT)
Zoom (Mouse wheel + SHIFT)
Output
By default a window is opened to display the rendering, but the extension can also be run in headless mode with the
headless
parameter.Using a display in exclusive mode is also supported with the
use_exclusive_display
parameter. This reduces the latency by avoiding the desktop compositor.The rendered framebuffer can be output to
render_buffer_output
.Public Types
-
enum class InputType
Input type.
All geometric primitives expect a 1d array of coordinates. Coordinates range from 0.0 (left, top) to 1.0 (right, bottom).
Values:
- enumerator UNKNOWN
unknown type, the operator tries to guess the type by inspecting the tensor
- enumerator COLOR
RGB or RGBA color 2d image.
- enumerator COLOR_LUT
single channel 2d image, color is looked up
- enumerator POINTS
point primitives, one coordinate (x, y) per primitive
- enumerator LINES
line primitives, two coordinates (x0, y0) and (x1, y1) per primitive
- enumerator LINE_STRIP
line strip primitive, a line primitive i is defined by each coordinate (xi, yi) and the following (xi+1, yi+1)
- enumerator TRIANGLES
triangle primitive, three coordinates (x0, y0), (x1, y1) and (x2, y2) per primitive
- enumerator CROSSES
cross primitive, a cross is defined by the center coordinate and the size (xi, yi, si)
- enumerator RECTANGLES
axis aligned rectangle primitive, each rectangle is defined by two coordinates (xi, yi) and (xi+1, yi+1)
- enumerator OVALS
oval primitive, an oval primitive is defined by the center coordinate and the axis sizes (xi, yi, sxi, syi)
- enumerator TEXT
text is defined by the top left coordinate and the size (x, y, s) per string, text strings are define by InputSpec::text_
- enumerator DEPTH_MAP
single channel 2d array where each element represents a depth value. The data is rendered as a 3d object using points, lines or triangles. The color for the elements can be specified through
DEPTH_MAP_COLOR
. Supported format: 8-bit unsigned normalized format that has a single 8-bit depth component- enumerator DEPTH_MAP_COLOR
RGBA 2d image, same size as the depth map. One color value for each element of the depth map grid. Supported format: 32-bit unsigned normalized format that has an 8-bit R component in byte > 0, an 8-bit G component in byte 1, an 8-bit B component in byte 2, and an 8-bit A component in byte 3unsigned 8-bit RGBA
-
enum class DepthMapRenderMode
Depth map render mode.
Values:
- enumerator POINTS
render points
- enumerator LINES
render lines
- enumerator TRIANGLES
render triangles
Public Functions
- HOLOSCAN_OPERATOR_FORWARD_ARGS (HolovizOp) HolovizOp()=default
-
virtual void setup(OperatorSpec &spec) override
Define the operator specification.
- Parameters
spec – The reference to the operator specification.
- virtual void initialize() override
Initialize the operator.
This function is called after the operator is created by holoscan::Fragment::make_operator().
- virtual void start() override
Implement the startup logic of the operator.
This method is called multiple times over the lifecycle of the operator according to the order defined in the lifecycle, and used for heavy initialization tasks such as allocating memory resources.
-
virtual void compute(InputContext &op_input, OutputContext &op_output, ExecutionContext &context) override
Implement the compute method.
This method is called by the runtime multiple times. The runtime calls this method until the operator is stopped.
- Parameters
op_input – The input context of the operator.
op_output – The output context of the operator.
context – The execution context of the operator.
- virtual void stop() override
Implement the shutdown logic of the operator.
This method is called multiple times over the lifecycle of the operator according to the order defined in the lifecycle, and used for heavy deinitialization tasks such as deallocation of all resources previously assigned in start.
-
struct InputSpec
Input specification
Public Functions
- InputSpec() = default
- inline InputSpec(const std::string tensor_name, InputType type)
-
inline explicit operator bool() const noexcept
- Returns
true if the input spec is valid
Public Members
- std::string tensor_name_
name of the tensor containing the input data
- float opacity_ = 1.f
layer opacity, 1.0 is fully opaque, 0.0 is fully transparent
- int32_t priority_ = 0
layer priority, determines the render order, layers with higher priority values are rendered on top of layers with lower priority values
- std::vector<float> color_ = {1.f, 1.f, 1.f, 1.f}
color of rendered geometry
- float line_width_ = 1.f
line width for geometry made of lines
- float point_size_ = 1.f
point size for geometry made of points
- std::vector<std::string> text_
array of text strings, used when type_ is TEXT.
- DepthMapRenderMode depth_map_render_mode_ = DepthMapRenderMode::POINTS
depth map render mode, used if type_ is DEPTH_MAP or DEPTH_MAP_COLOR.