# Creating Operators

## C++ Operators

When assembling a C++ application, two types of operators can be used:

1. Native C++ operators: custom operators defined in C++ without using the GXF API, by creating a subclass of holoscan::Operator. These C++ operators can pass arbitrary C++ shared objects around between operators.

2. GXF Operators: operators defined in the underlying C++ library by inheriting from the holoscan::ops::GXFOperator class. These operators wrap GXF codelets from GXF extensions. Examples are VideoStreamReplayerOp for replaying video files, FormatConverterOp for format conversions, and HolovizOp for visualization.

Note

It is possible to create an application using a mixture of GXF operators and native operators. In this case, some special consideration to cast the input and output tensors appropriately must be taken, as shown in a section below.

### Native C++ Operators

#### Operator Lifecycle (C++)

The lifecycle of a holoscan::Operator is made up of three stages:

• start() is called once when the operator starts, and is used for initializing heavy tasks such as allocating memory resources and using parameters.

• compute() is called when the operator is triggered, which can occur any number of times throughout the operator lifecycle between start() and stop().

• stop() is called once when the operator is stopped, and is used for deinitializing heavy tasks such as deallocating resources that were previously assigned in start().

All operators on the workflow are scheduled for execution. When an operator is first executed, the start() method is called, followed by the compute() method. When the operator is stopped, the stop() method is called. The compute() method is called multiple times between start() and stop().

If any of the scheduling conditions specified by Conditions are not met (for example, the CountCondition would cause the scheduling condition to not be met if the operator has been executed a certain number of times), the operator is stopped and the stop() method is called.

We will cover how to use Conditions in Configuring operator inputs and outputs (C++) and Adding conditions to operators (C++) sections of the user guide.

Typically, the start() and the stop() functions are only called once during the application’s lifecycle. However, if the scheduling conditions are met again, the operator can be scheduled for execution, and the start() method will be called again.

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% flowchart LR start(start) stop(stop) compute(compute) start --> compute compute --> compute compute --> stop

Fig. 10 The sequence of method calls in the lifecycle of a Holoscan Operator

We can override the default behavior of the operator by implementing the above methods. The following example shows how to implement a custom operator that overrides start, stop and compute methods.

Listing 6 The basic structure of a Holoscan Operator (C++)

Copy
Copied!

#include "holoscan/holoscan.hpp"

using holoscan::Operator;
using holoscan::OperatorSpec;
using holoscan::InputContext;
using holoscan::OutputContext;
using holoscan::ExecutionContext;
using holoscan::Arg;
using holoscan::ArgList;

class MyOp : public Operator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS(MyOp)

MyOp() = default;

void setup(OperatorSpec& spec) override {
}

void start() override {
HOLOSCAN_LOG_TRACE("MyOp::start()");
}

void compute(InputContext&, OutputContext& op_output, ExecutionContext&) override {
HOLOSCAN_LOG_TRACE("MyOp::compute()");
};

void stop() override {
HOLOSCAN_LOG_TRACE("MyOp::stop()");
}
};


#### Creating a custom operator (C++)

To create a custom operator in C++ it is necessary to create a subclass of holoscan::Operator. The following example demonstrates how to use native operators (the operators that do not have an underlying, pre-compiled GXF Codelet).

Code Snippet: examples/native_operator/cpp/ping.cpp

Listing 7 examples/native_operator/cpp/ping.cpp

Copy
Copied!

#include "holoscan/holoscan.hpp"

class ValueData {
public:
ValueData() = default;
explicit ValueData(int value) : data_(value) {
HOLOSCAN_LOG_TRACE("ValueData::ValueData(): {}", data_);
}
~ValueData() {
HOLOSCAN_LOG_TRACE("ValueData::~ValueData(): {}", data_);
}

void data(int value) { data_ = value; }

int data() const { return data_; }

private:
int data_;
};

namespace holoscan::ops {

class PingTxOp : public Operator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS(PingTxOp)

PingTxOp() = default;

void setup(OperatorSpec& spec) override {
spec.output<ValueData>("out1");
spec.output<ValueData>("out2");
}

void compute(InputContext&, OutputContext& op_output, ExecutionContext&) override {
auto value1 = std::make_shared
<valuedata>
(index_++);
</valuedata> op_output.emit(value1, "out1");
auto value2 = std::make_shared<ValueData>(index_++);
op_output.emit(value2, "out2");
};
int index_ = 0;
};

class PingMiddleOp : public Operator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS(PingMiddleOp)

PingMiddleOp() = default;

void setup(OperatorSpec& spec) override {
spec.input<ValueData>("in1");
spec.input<ValueData>("in2");
spec.output<ValueData>("out1");
spec.output<ValueData>("out2");
spec.param(multiplier_, "multiplier", "Multiplier", "Multiply the input by this value", 2);   }

void compute(InputContext& op_input, OutputContext& op_output, ExecutionContext&) override {
auto value1 = op_input.receive
<valuedata>
("in1");
</valuedata>    auto value2 = op_input.receive<ValueData>("in2");

HOLOSCAN_LOG_INFO("Middle message received (count: {})", count_++);

HOLOSCAN_LOG_INFO("Middle message value1: {}", value1->data());
HOLOSCAN_LOG_INFO("Middle message value2: {}", value2->data());

// Multiply the values by the multiplier parameter
value1->data(value1->data() * multiplier_);
value2->data(value2->data() * multiplier_);

op_output.emit(value1, "out1");
op_output.emit(value2, "out2");
};

private:
int count_ = 1;
Parameter
<int>
multiplier_;
</int>};

class PingRxOp : public Operator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS(PingRxOp)

PingRxOp() = default;

void setup(OperatorSpec& spec) override {

void compute(InputContext& op_input, OutputContext&, ExecutionContext&) override {
auto value_vector = op_input.receive
<std::vector>
<valuedata>
</valuedata>
</std::vector>
HOLOSCAN_LOG_INFO("Rx message received (count: {}, size: {})", count_++, value_vector.size());

HOLOSCAN_LOG_INFO("Rx message value1: {}", value_vector[0]->data());
HOLOSCAN_LOG_INFO("Rx message value2: {}", value_vector[1]->data());
};

private:
Parameter
<std::vector>
<iospec*>
</iospec*>
</std::vector>  int count_ = 1;
};

}  // namespace holoscan::ops

class App : public holoscan::Application {
public:
void compose() override {
using namespace holoscan;

auto tx = make_operator
<ops::pingtxop>
("tx", make_condition
<countcondition>
(10));
</countcondition>
</ops::pingtxop> auto mx = make_operator
<ops::pingmiddleop>
("mx", from_config("mx"));
</ops::pingmiddleop>    auto rx = make_operator<ops::PingRxOp>("rx");

add_flow(tx, mx, {{"out1", "in1"}, {"out2", "in2"}});
}
};

int main(int argc, char** argv) {

auto app = holoscan::make_application<App>();

// Get the configuration
auto config_path = std::filesystem::canonical(argv[0]).parent_path();
config_path += "/app_config.yaml";
app->config(config_path);

app->run();

return 0;
}


Code Snippet: examples/native_operator/cpp/app_config.yaml

Listing 8 examples/native_operator/cpp/app_config.yaml

Copy
Copied!

extensions:
- libgxf_std.so

mx:
multiplier: 3


In this application, three operators are created: PingTxOp, PingMiddleOp, and PingRxOp

1. The PingTxOp operator is a source operator that emits two values every time it is invoked. The values are emitted on two different output ports, out1 (for even integers) and out2 (for odd integers).

2. The PingMiddleOp operator is a middle operator that receives two values from the PingTxOp operator and emits two values on two different output ports. The values are multiplied by the multiplier parameter.

3. The PingRxOp operator is a sink operator that receives two values from the PingMiddleOp operator. The values are received on a single input, receivers, which is a vector of input ports. The PingRxOp operator receives the values in the order they are emitted by the PingMiddleOp operator.

As covered in more detail below, the inputs to each operator are specified in the setup() method of the operator. Then inputs are received within the compute() method via op_input.receive() and outputs are emitted via op_output.emit().

Note that for native C++ operators as defined here, any shared pointer can be emitted or received. When trasmitting between operators, a shared pointer to the object is transmitted rather than a copy. In some cases, such as sending the same tensor to more than one downstream operator, it may be necessary to avoid in-place operations on the tensor in order to avoid any potential race conditions between operators.

#### Configuring operator parameters (C++)

Arguments for custom parameters can be provided either directly as ArgList/Arg object(s) or accessed from the application’s YAML configuration file using from_config() method.

In the example holoscan::ops::PingMiddleOp operator above, we have a parameter multiplier that is declared (line 94) and configured (line 72, through the spec.param()) in the setup() method.

Copy
Copied!

94: Parameter<int> multiplier_;
72: spec.param(multiplier_, "multiplier", "Multiplier", "Multiply the input by this value", 2);


By default, the parameter is configured to have a default value of 2.

The holoscan::Fragment::from_config() method which returns a ArgList object can be used to access the configuration. it takes a string argument that is the path (dot-separated) to the parameters as a key in the YAML configuration file.

In this case, the YAML configuration is named mx and is defined in the app_config.yaml file.

By executing the following statement in the compose() method, the parameter is overwritten with the value from the YAML configuration file and the multiplier parameter is configured to have a value of 3.

Copy
Copied!

129: auto mx = make_operator<ops::PingMiddleOp>("mx", from_config("mx"));


An operator is created with the holoscan::Fragment::make_operator()() method which returns a shared pointer (e.g., std::shared_ptr<ops::PingMiddleOp>) to the operator object. The first argument is the name of the operator (can be omitted). The remaining arguments can be any number of:

• ArgList

• Arg

• Condition(std::shared_ptr<holoscan::Condition>)

• Resource(std::shared_ptr<holoscan::Resource>)

objects.

We can also provide the parameter value directly as Arg object.

Copy
Copied!

129: auto mx = make_operator<ops::PingMiddleOp>("mx", Arg("multiplier") = 3);
or
129: auto mx = make_operator<ops::PingMiddleOp>("mx", Arg("multiplier", 3));


Multiple parameters can be configured in the same way. If multiple arguments with the same name are provided, the last one will be used.

When creating the first operator, PingTxOp, we have an argument make_condition<CountCondition>(10):

Copy
Copied!

128: auto tx = make_operator<ops::PingTxOp>("tx", make_condition<CountCondition>(10));


holoscan::Fragment::make_condition() is a helper function that returns a Condition object (std::shared_ptr<Condition>). CountCondition is a condition that will control the total number of times the compute() method on PingTxOp will be called. In this case, the operator will stop after 10 invocations.

#### Configuring operator inputs and outputs (C++)

To configure the input(s) and output(s) of C++ native operators, call the spec.input() and spec.output() methods within the setup() method of the operator.

The spec.input() and spec.output() methods should be called once for each input and output to be added. The OperatorSpec object and the setup() method will be initialized and called automatically by the Application class when its run() method is called.

These methods (spec.input() and spec.output()) return an IOSpec object that can be used to configure the input/output port.

By default, the holoscan::MessageAvailableCondition and holoscan::DownstreamMessageAffordableCondition conditions are applied (with a min_size of 1) to the input/output ports. This means that the operator’s compute() method will not be invoked until a message is available on the input port and the downstream operator’s input port (queue) has enough capacity to receive the message.

Copy
Copied!

  void setup(OperatorSpec& spec) override {
spec.input<ValueData>("in");
// Above statement is equivalent to:
// spec.input
<valuedata>
("in")
</valuedata>
// .condition(ConditionType::kMessageAvailable, Arg("min_size") = 1);

spec.output<ValueData>("out");
// Above statement is equivalent to:
// spec.output
<valuedata>
("out")
</valuedata>
// .condition(ConditionType::kDownstreamMessageAffordable, Arg("min_size") = 1);
...
}


In the above example, the spec.input() method is used to configure the input port to have the holoscan::MessageAvailableCondition with a minimum size of 1. This means that the operator’s compute() method will not be invoked until a message is available on the input port of the operator. Similarly, the spec.output() method is used to configure the output port to have the holoscan::DownstreamMessageAffordableCondition with a minimum size of 1. This means that the operator’s compute() method will not be invoked until the downstream operator’s input port has enough capacity to receive the message.

If you want to change this behavior, use the IOSpec::condition() method to configure the conditions. For example, to configure the input and output ports to have no conditions, you can use the following code:

Copy
Copied!

  void setup(OperatorSpec& spec) override {
spec.input<ValueData>("in")
.condition(ConditionType::kNone);

spec.output<ValueData>("out")
.condition(ConditionType::kNone);
...
}


The example code in the setup() method configures the input port to have no conditions, which means that the compute() method will be called as soon as the operator is ready to compute. Since there is no guarantee that the input port will have a message available, the compute() method should check if there is a message available on the input port before attempting to read it.

The receive() method of the InputContext object can be used to access different types of input data within the compute() method of your operator class, where its template argument (DataT) is the data type of the input. This method takes the name of the input port as an argument (which can be omitted if your operator has a single input port), and returns a shared pointer to the input data.

In the example code fragment below, the PingRxOp operator receives input on a port called “in” with data type ValueData. The receive() method is used to access the input data, and the data() method of the ValueData class is called to get the value of the input data.

Copy
Copied!

// ...

class PingRxOp : public holoscan::ops::GXFOperator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS_SUPER(PingRxOp, holoscan::ops::GXFOperator)
PingRxOp() = default;
void setup(OperatorSpec& spec) override {
spec.input<ValueData>("in");
}
void compute(InputContext& op_input, OutputContext&, ExecutionContext&) override {
// The type of value is std::shared_ptr
<valuedata>

</valuedata>
auto value = op_input.receive<ValueData>("in");
HOLOSCAN_LOG_INFO("Message received (value: {})", value->data());
}
};


For objects of type std::any, the receive() method will return a std::any object containing the input of the specified name. In the example below, the PingRxOp operator receives input on a port called “in” with data type std::any. The type() method of the std::any object is used to determine the actual type of the input data, and the std::any_cast<T>() function is used to retrieve the value of the input data.

Copy
Copied!

// ...

class PingRxOp : public holoscan::ops::GXFOperator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS_SUPER(PingRxOp, holoscan::ops::GXFOperator)
PingRxOp() = default;
void setup(OperatorSpec& spec) override {
spec.input<std::any>("in");
}
void compute(InputContext& op_input, OutputContext&, ExecutionContext&) override {
// The type of in_any is 'std::any'.
auto in_any = op_input.receive<std::any>("in");
if (in_any.type() == typeid(holoscan::gxf::Entity)) {
auto in_entity = std::any_cast<holoscan::gxf::Entity>(in_any);
// Process with in_entity.
// ...
} else if (in_any.type() == typeid(std::shared_ptr<ValueData>)) {
auto in_message = std::any_cast<std::shared_ptr<ValueData>>(in_any);
// Process with in_message.
// ...
} else {
HOLOSCAN_LOG_ERROR("Message is not available");
return;
}
}
};


The Holoscan SDK provides built-in data types called Domain Objects, defined in the include/holoscan/core/domain directory. For example, the holoscan::Tensor is a Domain Object class that is used to represent a multi-dimensional array of data, which can be used directly by OperatorSpec, InputContext, and OutputContext.

Tip

This holoscan::Tensor class is a wrapper around the DLManagedTensorCtx struct holding a DLManagedTensor object. As such, it provides a primary interface to access Tensor data and is interoperable with other frameworks that support the DLPack interface.

Warning

Passing holoscan::Tensor objects to/from GXF operators directly is not supported. Instead, they need to be passed through holoscan::gxf::Entity objects. See the interoperability section for more details.

#### Receiving any number of inputs (C++)

Instead of assigning a specific number of input ports, it may be desired to have the ability to receive any number of objects on a port in certain situations. This can be done by defining Parameter with std::vector<IOSpec*>> (Parameter<std::vector<IOSpec*>> receivers_) and calling spec.param(receivers_, "receivers", "Input Receivers", "List of input receivers.", {}); as done for PingRxOp in the native operator ping example.

Listing 9 examples/native_operator/cpp/ping.cpp

Copy
Copied!

class PingRxOp : public Operator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS(PingRxOp)

PingRxOp() = default;

void setup(OperatorSpec& spec) override {

void compute(InputContext& op_input, OutputContext&, ExecutionContext&) override {
auto value_vector = op_input.receive
<std::vector>
<valuedata>
</valuedata>
</std::vector>
HOLOSCAN_LOG_INFO("Rx message received (count: {}, size: {})", count_++, value_vector.size());

HOLOSCAN_LOG_INFO("Rx message value1: {}", value_vector[0]->data());
HOLOSCAN_LOG_INFO("Rx message value2: {}", value_vector[1]->data());
};

private:
Parameter
<std::vector>
<iospec*>
</iospec*>
</std::vector>  int count_ = 1;
};

}  // namespace holoscan::ops

class App : public holoscan::Application {
public:
void compose() override {
using namespace holoscan;

auto tx = make_operator<ops::PingTxOp>("tx", make_condition<CountCondition>(10));
auto mx = make_operator<ops::PingMiddleOp>("mx", from_config("mx"));
auto rx = make_operator<ops::PingRxOp>("rx");

add_flow(tx, mx, {{"out1", "in1"}, {"out2", "in2"}});
};


Then, once the following configuration is provided in the compose() method, the PingRxOp will receive two inputs on the receivers port.

Copy
Copied!

133: add_flow(mx, rx, {{"out1", "receivers"}, {"out2", "receivers"}});


By using a parameter (receivers) with std::vector<holoscan::IOSpec*> type, the framework creates input ports (receivers:0 and receivers:1) implicitly and connects them (and adds the references of the input ports to the receivers vector).

#### Adding conditions to operators (C++)

Condition classes defined in Conditions can be passed to the constructor of an operator. For example, CountCondition can be added so that the compute() method will only be called a specific number of times. An example of this is given in the ping.cpp example (examples/native_operator/cpp/ping.cpp) built using only C++ native operators. In that example, the transmitter (PingTxOp) has a CountCondition applied that limits the number of messages transmitted to 10.

Copy
Copied!

  void compose() override {
// ...
auto tx = make_operator<ops::PingTxOp>("tx", make_condition<CountCondition>(10));
// ...
}


Similarly, a BooleanCondition can be used to configure whether an operator is enabled or disabled.

Copy
Copied!

  void compose() override {
// ...
auto tx = make_operator<ops::PingTxOp>("tx", make_condition<BooleanCondition>("is_alive"));
// ...
}


The first argument to the make_condition() function is the name of the condition. The name is used to refer to the condition in compute() method like below.

Copy
Copied!

  void compute(InputContext&, OutputContext& op_output, ExecutionContext&) override {
// ...
if (<condition expression>) {  // e.g. if (index_ >= 10)
auto is_alive = condition<BooleanCondition>("is_alive");
if (is_alive) {  // if the boolean condition is defined
is_alive->disable_tick();
}
}
// ...
}


### GXF Operators

With the Holoscan C++ API, we can also wrap GXF Codelets from GXF extensions as Holoscan Operators.

Note

If you do not have an existing GXF extension, we recommend developing native operators using the C++ or Python APIs to skip the need for wrapping gxf codelets as operators. If you do need to create a GXF Extension, follow the Creating a GXF Extension section for a detailed explanation of the GXF extension development process.

Given an existing GXF extension, we can create a simple “identity” application consisting of a replayer, which reads contents from a file on disk, and our recorder from the last section, which will store the output of the replayer exactly in the same format. This allows us to see whether the output of the recorder matches the original input files.

The MyRecorderOp Holoscan Operator implementation below will wrap the MyRecorder GXF Codelet shown here:

Listing 10 my_recorder_op.hpp

Copy
Copied!

#ifndef APPS_MY_RECORDER_APP_MY_RECORDER_OP_HPP
#define APPS_MY_RECORDER_APP_MY_RECORDER_OP_HPP

#include "holoscan/core/gxf/gxf_operator.hpp"

namespace holoscan::ops {

class MyRecorderOp : public holoscan::ops::GXFOperator {
public:
HOLOSCAN_OPERATOR_FORWARD_ARGS_SUPER(MyRecorderOp, holoscan::ops::GXFOperator)

MyRecorderOp() = default;

const char* gxf_typename() const override { return "MyRecorder"; }

void setup(OperatorSpec& spec) override;

void initialize() override;

private:
Parameter
<holoscan::iospec*>
</holoscan::iospec*> Parameter
<std::shared_ptr>
<holoscan::resource>
> my_serializer_;
</holoscan::resource>
</std::shared_ptr>  Parameter<std::string> directory_;
Parameter<std::string> basename_;
Parameter<bool> flush_on_tick_;
};

}  // namespace holoscan::ops

#endif/* APPS_MY_RECORDER_APP_MY_RECORDER_OP_HPP */


The holoscan::ops::MyRecorderOp class wraps a MyRecorder GXF Codelet by inheriting from the holoscan::ops::GXFOperator class. The HOLOSCAN_OPERATOR_FORWARD_ARGS_SUPER macro is used to forward the arguments of the constructor to the base class.

We first need to define the fields of the MyRecorderOp class. You can see that fields with the same names are defined in both the MyRecorderOp class and the MyRecorder GXF codelet .

Listing 11 Parameter declarations in my_recorder_op.hpp

Copy
Copied!

 nvidia::gxf::Parameter
<nvidia::gxf::handle>
</nvidia::gxf::handle> nvidia::gxf::Parameter
<nvidia::gxf::handle>
<nvidia::gxf::entityserializer>
> my_serializer_;
</nvidia::gxf::entityserializer>
</nvidia::gxf::handle>  nvidia::gxf::Parameter<std::string> directory_;
nvidia::gxf::Parameter<std::string> basename_;
nvidia::gxf::Parameter<bool> flush_on_tick_;


Comparing the MyRecorderOp holoscan parameter to the MyRecorder gxf codelet:

Holoscan Operator

GXF Codelet

holoscan::Parameter

nvidia::gxf::Parameter

holoscan::IOSpec*

nvidia::gxf::Handle<nvidia::gxf::Receiver>>
or nvidia::gxf::Handle<nvidia::gxf::Transmitter>>

std::shared_ptr<holoscan::Resource>>

nvidia::gxf::Handle<T>>
example: T is nvidia::gxf::EntitySerializer

We then need to implement the following functions:

• const char* gxf_typename() const override: return the GXF type name of the Codelet. The fully-qualified class name (MyRecorder) for the GXF Codelet is specified.

• void setup(OperatorSpec& spec) override: setup the OperatorSpec with the inputs/outputs and parameters of the Operator.

• void initialize() override: initialize the Operator.

The implementation of the setup(OperatorSpec& spec) function is as follows:

Listing 12 my_recorder_op.cpp

Copy
Copied!

#include "./my_recorder_op.hpp"

#include "holoscan/core/fragment.hpp"
#include "holoscan/core/gxf/entity.hpp"
#include "holoscan/core/operator_spec.hpp"

#include "holoscan/core/resources/gxf/video_stream_serializer.hpp"

namespace holoscan::ops {

void MyRecorderOp::setup(OperatorSpec& spec) {
auto& input = spec.input
<holoscan::gxf::entity>
("input");
</holoscan::gxf::entity> // Above is same with the following two lines (a default condition is assigned to the input port if not specified):  //  // auto& input = spec.input
<holoscan::gxf::entity>
("input")
</holoscan::gxf::entity>  // .condition(ConditionType::kMessageAvailable, Arg("min_size") = 1);   spec.param(receiver_, "receiver", "Entity receiver", "Receiver channel to log", &input);   spec.param(my_serializer_,
"serializer",
"Entity serializer",
"Serializer for serializing input data");
spec.param(directory_, "out_directory", "Output directory path", "Directory path to store received output");
spec.param(basename_, "basename", "File base name", "User specified file name without extension");
spec.param(flush_on_tick_,
"flush_on_tick",
"Boolean to flush on tick",
"Flushes output buffer on every tick when true",
false);
}

void MyRecorderOp::initialize() {...}

}  // namespace holoscan::ops


Here, we set up the inputs/outputs and parameters of the Operator. Note how the content of this function is very similar to the MyRecorder GXF codelet’s registerInterface function.

• In the C++ API, GXF Receiver and Transmitter components (such as DoubleBufferReceiver and DoubleBufferTransmitter) are considered as input and output ports of the Operator so we register the inputs/outputs of the Operator with input<T> and output<T> functions (where T is the data type of the port).

• Compared to the pure GXF application that does the same job, the SchedulingTerm of an Entity in the GXF Application YAML are specified as Conditions on the input/output ports (e.g., holoscan::MessageAvailableCondition and holoscan::DownstreamMessageAffordableCondition).

The highlighted lines in MyRecorderOp::setup above match the following highlighted statements of GXF Application YAML:

Listing 13 A part of apps/my_recorder_app_gxf/my_recorder_gxf.yaml

Copy
Copied!

name: recorder
components:
- name: input  type: nvidia::gxf::DoubleBufferReceiver  - name: allocator
type: nvidia::gxf::UnboundedAllocator
- name: component_serializer
type: nvidia::gxf::StdComponentSerializer
parameters:
allocator: allocator
- name: entity_serializer
type: nvidia::holoscan::stream_playback::VideoStreamSerializer   # inheriting from nvidia::gxf::EntitySerializer
parameters:
component_serializers: [component_serializer]
- type: MyRecorder  parameters:  receiver: input      serializer: entity_serializer
out_directory: "/tmp"
basename: "tensor_out"
- type: nvidia::gxf::MessageAvailableSchedulingTerm  parameters:  receiver: input  min_size: 1 


In the same way, if we had a Transmitter GXF component, we would have the following statements (Please see available constants for holoscan::ConditionType):

Copy
Copied!

  auto& output = spec.output<holoscan::gxf::Entity>("output");
// Above is same with the following two lines (a default condition is assigned to the output port if not specified):
//
// auto& output = spec.output
<holoscan::gxf::entity>
("output")
</holoscan::gxf::entity>
// .condition(ConditionType::kDownstreamMessageAffordable, Arg("min_size") = 1);


Next, the implementation of the initialize() function is as follows:

Listing 14 my_recorder_op.cpp

Copy
Copied!

#include "./my_recorder_op.hpp"

#include "holoscan/core/fragment.hpp"
#include "holoscan/core/gxf/entity.hpp"
#include "holoscan/core/operator_spec.hpp"

#include "holoscan/core/resources/gxf/video_stream_serializer.hpp"

namespace holoscan::ops {

void MyRecorderOp::setup(OperatorSpec& spec) {...}

void MyRecorderOp::initialize() {
// Set up prerequisite parameters before calling GXFOperator::initialize()
auto frag = fragment();  auto serializer =  frag->make_resource
<holoscan::videostreamserializer>
("serializer");
</holoscan::videostreamserializer> add_arg(Arg("serializer") = serializer);
GXFOperator::initialize();
}

}  // namespace holoscan::ops


Here we set up the pre-defined parameters such as the serializer. The highlighted lines above matches the highlighted statements of GXF Application YAML:

Listing 15 Another part of apps/my_recorder_app_gxf/my_recorder_gxf.yaml

Copy
Copied!

name: recorder
components:
- name: input
- name: allocator  type: nvidia::gxf::UnboundedAllocator  - name: component_serializer  type: nvidia::gxf::StdComponentSerializer  parameters:  allocator: allocator  - name: entity_serializer  type: nvidia::holoscan::stream_playback::VideoStreamSerializer # inheriting from nvidia::gxf::EntitySerializer  parameters:  component_serializers: [component_serializer]  - type: MyRecorder
parameters:
serializer: entity_serializer      out_directory: "/tmp"
basename: "tensor_out"
- type: nvidia::gxf::MessageAvailableSchedulingTerm
parameters:
min_size: 1


Note

The Holoscan C++ API already provides the holoscan::VideoStreamSerializer class which wraps the nvidia::holoscan::stream_playback::VideoStreamSerializer GXF component, used here as serializer.

### Interoperability between GXF and native C++ operators

GXF passes nvidia::gxf::Tensor types between its codelets through a nvidia::gxf::Entity message. To support sending or receiving tensors to and from a GXF codelet (wrapped in a GXF operator) the Holoscan SDK provides the C++ classes below:

• holoscan::gxf::GXFTensor: inherits from nvidia::gxf::Tensor, and holds a DLManagedTensorCtx struct, making it interchangeable with the holoscan::Tensor class mentioned above.

• holoscan::gxf::Entity: inherits from nvidia::gxf::Entity, handles the conversion from holoscan::gxf::GXFTensor to holoscan::Tensor under the hood.

Fig. 11 Supporting Tensor Interoperability

Consider the following example, where GXFSendTensorOp and GXFReceiveTensorOp are GXF operators, and where ProcessTensorOp is a C++ native operator:

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% classDiagram direction LR GXFSendTensorOp --|> ProcessTensorOp : signal...in ProcessTensorOp --|> GXFReceiveTensorOp : out...signal class GXFSendTensorOp { signal(out) Tensor } class ProcessTensorOp { [in]in : Tensor out(out) Tensor } class GXFReceiveTensorOp { [in]signal : Tensor }

Fig. 12 The tensor interoperability between C++ native operator and GXF operator

The following code shows how to implement ProcessTensorOp’s compute() method as a C++ native operator communicating with GXF operators. Focus on the use of the holoscan::gxf::Entity:

Listing 16 examples/tensor_interop/cpp/tensor_interop.cpp

Copy
Copied!

void compute(InputContext& op_input, OutputContext& op_output,
ExecutionContext& context) override {
// The type of in_message is 'holoscan::gxf::Entity'.
auto in_message = op_input.receive
<holoscan::gxf::entity>
("in");
</holoscan::gxf::entity>  // The type of tensor is 'std::shared_ptr
<holoscan::tensor>
'.
</holoscan::tensor>
auto tensor = in_message.get
<tensor>
();
</tensor>
// Process with 'tensor' here.
cudaError_t cuda_status;

size_t data_size = tensor->nbytes();
std::vector<uint8_t> in_data(data_size);
CUDA_TRY(cudaMemcpy(in_data.data(), tensor->data(), data_size, cudaMemcpyDeviceToHost));

for (size_t i = 0; i < data_size; i++) { in_data[i] *= 2; }

CUDA_TRY(cudaMemcpy(tensor->data(), in_data.data(), data_size, cudaMemcpyHostToDevice));

// Create a new message (Entity)
auto out_message = holoscan::gxf::Entity::New(&context);  out_message.add(tensor, "tensor");
// Send the processed message.
op_output.emit(out_message); };


• The op_input.receive() method call returns a holoscan::gxf::Entity object. That object has a get() method that returns the holoscan::Tensor object.

• The holoscan::Tensor object is copied on the host as in_data.

• The data is process (values multiplied by 2)

• The data is moved back to the holoscan::Tensor object on the GPU.

• A new holoscan::gxf::Entity object is created to be sent to the next operator with op_output.emit(). The holoscan::Tensor object is added to it using the add() method.

Note

A complete example of the C++ native operator that supports interoperability with GXF operators is available in the examples/tensor_interop/cpp directory.

You can add multiple tensors to a single holoscan::gxf::Entity object by calling the add() method multiple times with a unique name for each tensor, as in the example below:

Operator sending a message:

Copy
Copied!

    auto out_message = holoscan::gxf::Entity::New(&context);
// Tensors and tensor names
// Entity and port name
op_output.emit(out_message, "outputs");


Operator receiving the message, assuming the outputs port above is connected to the inputs port below with add_flow():

Copy
Copied!

// Entity and port name
auto in_message = op_input.receive<holoscan::gxf::Entity>("inputs");
// Tensors and tensor names
auto video = in_message.get<Tensor>("video");
auto labels = in_message.get<Tensor>("labels");
auto bbox_coords = in_message.get<Tensor>("bbox_coords");    


Note

Some existing operators allow configuring the name of the tensors they send/receive. An example is the tensors parameter of HolovizOp, where the name for each tensor maps to the names of the tensors in the Entity (see the holoviz entry in apps/endoscopy_tool_tracking/cpp/app_config.yaml).

## Python Operators

When assembling a Python application, two types of operators can be used:

1. Native Python operators: custom operators defined in Python, by creating a subclass of holoscan.core.Operator. These Python operators can pass arbitrary Python objects around between operators and are not restricted to the stricter parameter typing used for C++ API operators.

2. Python wrappings of C++ Operators: operators defined in the underlying C++ library by inheriting from the holoscan::Operator class. These operators have Python bindings available within the holoscan.operators module. Examples are VideoStreamReplayerOp for replaying video files, FormatConverterOp for format conversions, and HolovizOp for visualization.

Note

It is possible to create an application using a mixture of Python wrapped C++ operators and native Python operators. In this case, some special consideration to cast the input and output tensors appropriately must be taken, as shown in a section below.

### Native Python Operator

#### Operator Lifecycle (Python)

The lifecycle of a holoscan.core.Operator is made up of three stages:

• start() is called once when the operator starts, and is used for initializing heavy tasks such as allocating memory resources and using parameters.

• compute() is called when the operator is triggered, which can occur any number of times throughout the operator lifecycle between start() and stop().

• stop() is called once when the operator is stopped, and is used for deinitializing heavy tasks such as deallocating resources that were previously assigned in start().

All operators on the workflow are scheduled for execution. When an operator is first executed, the start() method is called, followed by the compute() method. When the operator is stopped, the stop() method is called. The compute() method is called multiple times between start() and stop().

If any of the scheduling conditions specified by Conditions are not met (for example, the CountCondition would cause the scheduling condition to not be met if the operator has been executed a certain number of times), the operator is stopped and the stop() method is called.

We will cover how to use Conditions in Configuring operator inputs and outputs (Python) and Adding conditions to operators (Python) section of the user guide.

Typically, the start() and the stop() functions are only called once during the application’s lifecycle. However, if the scheduling conditions are met again, the operator can be scheduled for execution, and the start() method will be called again.

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% flowchart LR start(start) stop(stop) compute(compute) start --> compute compute --> compute compute --> stop

Fig. 13 The sequence of method calls in the lifecycle of a Holoscan Operator

We can override the default behavior of the operator by implementing the above methods. The following example shows how to implement a custom operator that overrides start, stop and compute methods.

Listing 17 The basic structure of a Holoscan Operator (Python)

Copy
Copied!

from holoscan.core import (
ExecutionContext,
InputContext,
Operator,
OperatorSpec,
OutputContext,
)

class MyOp(Operator):

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)

def setup(self, spec: OperatorSpec):
pass

def start(self):
pass

def compute(self, op_input: InputContext, op_output: OutputContext, context: ExecutionContext):
pass

def stop(self):
pass


#### Creating a custom operator (Python)

To create a custom operator in Python it is necessary to create a subclass of holoscan.core.Operator. A simple example of an operator that takes a time-varying 1D input array named “signal” and applies convolution with a boxcar (i.e. rect) kernel.

For simplicity, this operator assumes that the “signal” that will be received on the input is already a numpy.ndarray or is something that can be cast to one via (np.asarray). We will see more details in a later section on how we can interoperate with various tensor classes, including the GXF Tensor objects used by some of the C++-based operators.

Code Snippet: examples/native_operator/python/convolve.py

Listing 18 examples/native_operator/python/convolve.py

Copy
Copied!

import os

from holoscan.conditions import CountCondition
from holoscan.core import Application, Operator, OperatorSpec
from holoscan.logger import LogLevel, set_log_level

import numpy as np

class SignalGeneratorOp(Operator):
"""Generate a time-varying impulse.

Transmits an array of zeros with a single non-zero entry of a
specified height. The position of the non-zero entry shifts
to the right (in a periodic fashion) each time compute is
called.

Parameters
----------
height : number
The height of the signal impulse.
size : number
The total number of samples in the generated 1d signal.
dtype : numpy.dtype or str
The data type of the generated signal.
"""

def __init__(self, *args, height=1, size=10, dtype=np.int32, **kwargs):
self.count = 0
self.height = height
self.dtype = dtype
self.size = size
super().__init__(*args, **kwargs)

def setup(self, spec: OperatorSpec):
spec.output("signal")

def compute(self, op_input, op_output, context):

# single sample wide impulse at a time-varying position
signal = np.zeros((self.size,), dtype=self.dtype)
signal[self.count % signal.size] = self.height
self.count += 1

op_output.emit(signal, "signal")

class ConvolveOp(Operator):
"""Apply convolution to a tensor.

Convolves an input signal with a "boxcar" (i.e. "rect") kernel.

Parameters
----------
width : number
The width of the boxcar kernel used in the convolution.
unit_area : bool, optional
Whether or not to normalize the convolution kernel to unit area.
If False, all samples have implitude one and the dtype of the
kernel will match that of the signal. When True the sum over
the kernel is one and a 32-bit floating point data type is used
for the kernel.
"""

def __init__(self, *args, width=4, unit_area=False, **kwargs):
self.count = 0
self.width = width
self.unit_area = unit_area
super().__init__(*args, **kwargs)

def setup(self, spec: OperatorSpec):
spec.input("signal_in")
spec.output("signal_out")

def compute(self, op_input, op_output, context):

assert isinstance(signal, np.ndarray)

if self.unit_area:
kernel = np.full((self.width,), 1/self.width, dtype=np.float32)
else:
kernel = np.ones((self.width,), dtype=signal.dtype)

convolved = np.convolve(signal, kernel, mode='same')

op_output.emit(convolved, "signal_out")

class PrintSignalOp(Operator):
"""Print the received signal to the terminal."""

def setup(self, spec: OperatorSpec):
spec.input("signal")

def compute(self, op_input, op_output, context):
print(signal)

class ConvolveApp(Application):
"""Minimal signal processing application.

Generates a time-varying impulse, convolves it with a boxcar kernel, and
prints the result to the terminal.

A CountCondition is applied to the generate to terminate execution
after a specific number of steps.
"""

def compose(self):
signal_generator = SignalGeneratorOp(
self,
CountCondition(self, count=24),
name="generator",
**self.kwargs("generator"),
)
convolver = ConvolveOp(self, name="conv", **self.kwargs("convolve"))
printer = PrintSignalOp(self, name="printer")

if __name__ == "__main__":
set_log_level(LogLevel.WARN)

app = ConvolveApp()
config_file = os.path.join(os.path.dirname(__file__), 'convolve.yaml')
app.config(config_file)
app.run()


Code Snippet: examples/native_operator/python/convolve.yaml

Listing 19 examples/native_operator/python/convolve.yaml

Copy
Copied!

signal_generator:
height: 1
size: 20
dtype: int32

convolve:
width: 4
unit_area: false


In this application, three native Python operators are created: SignalGeneratorOp, ConvolveOp and PrintSignalOp. The SignalGeneratorOp generates a synthetic signal such as [0, 0, 1, 0, 0, 0] where the position of the non-zero entry varies each time it is called. ConvolveOp performs a 1D convolution with a boxcar (i.e. rect) function of a specified width. PrintSignalOp just prints the received signal to the terminal.

As covered in more detail below, the inputs to each operator are specified in the setup() method of the operator. Then inputs are received within the compute method via op_input.receive() and outputs are emitted via op_output.emit().

Note that for native Python operators as defined here, any Python object can be emitted or received. When trasmitting between operators, a shared pointer to the object is transmitted rather than a copy. In some cases, such as sending the same tensor to more than one downstream operator, it may be necessary to avoid in-place operations on the tensor in order to avoid any potential race conditions between operators.

#### Configuring operator parameters (Python)

Keyword arguments for custom parameters can be provided either directly as a Python kwarg or accessed from the application’s YAML configuration file. In the example SignalGeneratorOp operator above, we added three keyword arguments in the operator’s __init__ method. These parameters have default values, but can be specified to change the behavior of the operator. If we initialize the operator as follows, the defaults would be assigned:

Copy
Copied!

generator_op = SignalGeneratorOp(self, name="generator")


Note that the first argument to the operator is always the application to which the operator belongs. Because we configure the operators within the compose method of the application, self is always passed for this first argument to all operators.

To override the default parameter value, parameters can be provided either via a Python kwargs:

Copy
Copied!

generator_op = SignalGeneratorOp(self, name="generator", height=1.5, size=10, dtype=np.float32)


or if the YAML file passed to the application’s config method had a section such as:

Copy
Copied!

signal_generator:
height: 1
size: 20
dtype: int32


the parameters from the file could be accessed via

Copy
Copied!

generator_op = SignalGeneratorOp(self, name="generator", **self.kwargs("signal_generator"))


where the kwargs method of the Application class is used to get a Python dictionary of arguments corresponding to a specific name in the YAML file. When reading from YAML files, types are inferred similar to when reading from YAML via the C++ API. For example, if no arguments are to be read from a config file, the call to app.config can be omitted (or an empty string: app.config("") can be specified to avoid a logged warning about an unspecified configuration).

You can mix and match the two approaches unless the same argument is specified by both methods. (Otherwise, you will get an error message such as TypeError: SignalGeneratorOp() got multiple values for keyword argument 'height'.)

Taking a closer look at the __init__ method:

Copy
Copied!

    def __init__(self, *args, height=1, size=10, dtype=np.int32, **kwargs):
self.count = 0
self.height = height
self.dtype = dtype
self.size = size
super().__init__(*args, **kwargs)


We see that there is also a fourth parameter, count that is not exposed to the user. This is used as an internal counter for the time-dependent signal generation behavior in the compute() method of the operator.

It is also important to keep *args and **kwargs both in the signature and call super().__init__(*args, **kwargs) at the end of the constructor as shown. This ensures that one can pass other arguments to the underlying C++ Operator class. For example, in this demo application, the CountCondition object is passed to the underlying C++ class, SignalGeneratorOp:

Copy
Copied!

        signal_generator = SignalGeneratorOp(
self,
CountCondition(self, count=24),
name="generator",
**self.kwargs("signal_generator"),
)


This count condition will control the total number of times the compute() method on SignalGeneratorOp will be called.

#### Configuring operator inputs and outputs (Python)

To configure the input(s) and output(s) of Python native operators, call the spec.input() and spec.output() methods within the setup() method of the operator.

The spec.input() and spec.output() methods should be called once for each input and output to be added. The holoscan.core.OperatorSpec object and the setup() method will be initialized and called automatically by the Application class when its run() method is called.

These methods (spec.input() and spec.output()) return an IOSpec object that can be used to configure the input/output port.

By default, the holoscan.conditions.MessageAvailableCondition and holoscan.conditions.DownstreamMessageAffordableCondition conditions are applied (with a min_size of 1) to the input/output ports. This means that the operator’s compute() method will not be invoked until a message is available on the input port and the downstream operator’s input port (queue) has enough capacity to receive the message.

The receive() method of the InputContext object can be used to access different types of input data within the compute() method of your operator class. This method takes the name of the input port as an argument (which can be omitted if your operator has a single input port).

For standard Python objects, receive() will directly return the Python object for input of the specified name.

The Holoscan SDK also provides built-in data types called Domain Objects, defined in the include/holoscan/core/domain directory. For example, the Tensor is a Domain Object class that is used to represent a multi-dimensional array of data, which can be used directly by OperatorSpec, InputContext, and OutputContext.

Tip

This holoscan.core.Tensor class supports both DLPack and NumPy’s array interface (__array_interface__ and __cuda_array_interface__) so that it can be used with other Python libraries such as CuPy, PyTorch, JAX, TensorFlow, and Numba.

Warning

Passing holoscan.core.Tensor objects to/from Python wrapped C++ operators (both C++ native and GXF-based) directly is not yet supported. At this time, they need to be passed through holoscan.gxf.Entity objects. See the interoperability section for more details. This won’t be necessary in the future for native C++ operators.

#### Receiving any number of inputs (Python)

Instead of assigning a specific number of input ports, it may be desired to have the ability to receive any number of objects on a port in certain situations. This can be done by calling spec.param(port_name, kind='receivers') as done for PingRxOp in the native operator ping example located at examples/native_operator/python/ping.py:

Code Snippet: examples/native_operator/python/ping.py

Listing 20 examples/native_operator/python/ping.py

Copy
Copied!

class PingRxOp(Operator):

This operator has:

This is an example of a native operator that can dynamically have any
number of inputs connected to is "receivers" port.
"""

def __init__(self, *args, **kwargs):
self.count = 1
# Need to call the base class constructor last
super().__init__(*args, **kwargs)

def setup(self, spec: OperatorSpec):

def compute(self, op_input, op_output, context):
print(f"Rx message received (count:{self.count}, size:{len(values)})")
self.count += 1
print(f"Rx message value1:{values[0].data}")
print(f"Rx message value2:{values[1].data}")


and in the compose method of the application, two parameters are connected to this “receivers” port:

Copy
Copied!

        self.add_flow(mx, rx, {("out1", "receivers"), ("out2", "receivers")})


This line connects both the out1 and out2 ports of operator mx to the receivers port of operator rx.

Here, values as returned by op_input.receive("receivers") will be a tuple of python objects.

#### Adding conditions to operators (Python)

Condition classes defined in holoscan.conditions can be passed to the constructor of an operator. For example, CountCondition can be added so that the compute() method will only be called a specific number of times. An example of this is given in the ping.py example built using only Python native operators. In that example, the transmitter (PingTxOp) has a CountCondition applied that limits the number of messages transmitted to 10.

Copy
Copied!

    def compose(self):
# ...
my_op = MyOp(self, CountCondition(self, 10), name="my_op")
# ...


Similarly, a BooleanCondition can be used to configure whether an operator is enabled or disabled.

Copy
Copied!

    def compose(self):
# ...
my_op = MyOp(self, BooleanCondition(self, name="is_alive"), name="my_op")
# ...


The name of the BooleanCondition is used to refer to the condition in the compute() method like below.

Copy
Copied!

    def compute(self, op_input, op_output, context):
# ...
if <condition expression>:  # e.g, self.index >= 10
self.conditions["is_alive"].disable_tick()
# ...


The following code snippet shows how to use the BooleanCondition to enable/disable an operator.

Listing 21 How to use BooleanCondition in Python

Copy
Copied!

from holoscan.conditions import BooleanCondition
from holoscan.core import Application, Operator

class IsolatedOp(Operator):
def __init__(self, *args, **kwargs):
self.count = 0
super().__init__(*args, **kwargs)

def compute(self, op_input, op_output, context):
self.count += 1
if self.count == 1:
print(f"name:{self.name}")
print(f"conditions:{self.conditions}")

if self.count >= 10:
self.conditions["enabled"].disable_tick()

print(f"count ={self.count}")

class MinimalApp(Application):
def compose(self):
tx = IsolatedOp(self, BooleanCondition(self, name="enabled"), name="tx")

if __name__ == "__main__":
app = MinimalApp()
app.config("")
app.run()


### Python wrapping of a C++ operator

At this time, please refer to the existing python bindings in the source repository at python/pybind11/operators and python/CMakeLists.txt for reference.

### Interoperability between wrapped and native Python operators

As described in the Interoperability between GXF and native C++ operators section, holoscan::Tensor objects can only be passed to GXF operators using a holoscan::gxf::Entity message that holds the tensor(s). In Python, this is done with the wrapped methods, holoscan.core.Tensor and holoscan.gxf.Entity.

Warning

At this time, using holoscan.gxf.Entity is required when communicating with any Python wrapped C++ operator. That includes native C++ operators and GXF operators. This will be addressed in future versions to only require a holoscan.gxf.Entity for Python wrapped GXF operators.

Consider the following example, where VideoStreamReplayerOp and HolovizOp are Python wrapped C++ operators, and where ImageProcessingOp is a Python native operator:

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% classDiagram direction LR VideoStreamReplayerOp --|> ImageProcessingOp : output...input_tensor ImageProcessingOp --|> HolovizOp : output_tensor...receivers class VideoStreamReplayerOp { output_tensor(out) Tensor } class ImageProcessingOp { [in]input_tensor : Tensor output_tensor(out) Tensor } class HolovizOp { [in]receivers : Tensor }

Fig. 14 The tensor interoperability between Python native operator and C++-based Python GXF operator

The following code shows how to implement ImageProcessingOp’s compute() method as a Python native operator communicating with C++ operators:

Listing 22 examples/tensor_interop/python/tensor_interop.py

Copy
Copied!

    def compute(self, op_input, op_output, context):
input_tensor = message.get()
self.count += 1

cp_array = cp.asarray(input_tensor)
# smooth along first two axes, but not the color channels
sigma = (self.sigma, self.sigma, 0)

# process cp_array
cp_array = ndi.gaussian_filter(cp_array, sigma)

out_message = Entity(context)  output_tensor = hs.as_tensor(cp_array)
out_message.add(output_tensor)  op_output.emit(out_message, "output_tensor") 


• The op_input.receive() method call returns a holoscan.gxf.Entity object. That object has a get() method that returns a holoscan.core.Tensor object.

• The holoscan.core.Tensor object is converted to a CuPy array by using cupy.asarray() method call.

• The CuPy array is used as an input to the ndi.gaussian_filter() function call with a parameter sigma. The result of the ndi.gaussian_filter() function call is a CuPy array.

• The CuPy array is converted to a holoscan.core.Tensor object by using holoscan.as_tensor() function call.

• Finally, a new holoscan.gxf.Entity object is created to be sent to the next operator with op_output.emit(). The holoscan.core.Tensor object is added to it using the add() method.

Note

A complete example of the Python native operator that supports interoperability with Python wrapped C++ operators is available in the examples/tensor_interop/python directory.

You can add multiple tensors to a single holoscan.gxf.Entity object by calling the add() method multiple times with a unique name for each tensor, as in the example below:

Operator sending a message:

Copy
Copied!

    out_message = Entity(context)
# Tensors and tensor names
# Entity and port name
op_output.emit(out_message, "outputs")


Operator receiving the message, assuming the outputs port above is connected to the inputs port below with add_flow():

Copy
Copied!

# Entity and port name
# Tensors and tensor names
video = in_message.get("video")
labels = in_message.get("labels")
bbox_coords = in_message.get("bbox_coords")    


Note

Some existing operators allow configuring the name of the tensors they send/receive. An example is the tensors parameter of HolovizOp, where the name for each tensor maps to the names of the tensors in the Entity (see the holoviz entry in apps/endoscopy_tool_tracking/python/endoscopy_tool_tracking.yaml).

© Copyright 2022, NVIDIA. Last updated on Mar 20, 2023.