holoscan::ops::FormatConverterOp
holoscan::ops::FormatConverterOp
Documentationholoscan::ops::FormatConverterOp
Operator class to convert the data format of the input data.
==Named Inputs==
nvidia::gxf::Tensor or nvidia::gxf::VideoBuffer
The input video frame to process. If the input is a VideoBuffer it must be in format GXF_VIDEO_FORMAT_RGBA, GXF_VIDEO_FORMAT_RGB or GXF_VIDEO_FORMAT_NV12, GXF_VIDEO_FORMAT_NV12_ER, GXF_VIDEO_FORMAT_NV12_709 or GXF_VIDEO_FORMAT_NV12_709_ER. If a video buffer is not found, the input port message is searched for a tensor with the name specified by in_tensor_name. This must be a tensor in one of several supported formats (unsigned 8-bit int or float32 graycale, unsigned 8-bit int RGB or RGBA YUV420, NV12BT601Full, NV12BT709CSC, NV12BT709HDTV, or YUYV). The tensor or video buffer may be in either host or device memory (a host->device copy is performed if needed).
in_tensor_name. This must be a tensor in one of several supported formats (unsigned 8-bit int or float32 graycale, unsigned 8-bit int RGB or RGBA YUV420, NV12BT601Full, NV12BT709CSC, NV12BT709HDTV, or YUYV). The tensor or video buffer may be in either host or device memory (a host->device copy is performed if needed).==Named Outputs==
nvidia::gxf::Tensor
The output video frame after processing. The shape, data type and number of channels of this output tensor will depend on the specific parameters that were set for this operator. The name of the Tensor transmitted on this port is determined by out_tensor_name.
out_tensor_name.==Parameters==
"rgb888""uint8""float32""rgba8888""yuv420""rgb888""uint8""float32""rgba8888""yuv420""nv12" (alias for nv12_bt709_hdtv)"nv12_bt601_full" (BT.601 full range as in NPP’s NV12ToRGB functions and IPP’s YUVToRGB)"nv12_bt709_hdtv" (BT.709 recommendation for high-definition TV (HDTV))"nv12_bt709_csc" (BT.709 recommendation for computer systems consideration (CSC))"yuyv""rgb161616""rgba16161616" Optional (default: "rgb888")."")."").0.0).1.0).255).resize_height is 0. Optional (default: 0).resize_width is 0. Optional (default: 0).0): Undefined filtering interpolation mode.1): Nearest neighbor filtering.2): Linear interpolation.4): Cubic interpolation.5): Two-parameter cubic filter (B=1, C=0)6): Two-parameter cubic filter (B=0, C=1/2)7): Two-parameter cubic filter (B=1/2, C=3/10)8): Super sampling.16): Lanczos filtering.17): Generic Lanczos filtering with order 3.0x8000000): Smooth edge filtering.
Optional (default: 0). The default value 0 (NPPI_INTER_UNDEFINED) which would be equivalent to 4 (NPPI_INTER_CUBIC).[0, 1, 2] for 3-channel images and [0, 1, 2, 3] for 4-channel images).holoscan::CudaStreamPool instance to allocate CUDA streams. Optional (default: nullptr).==Device Memory Requirements==
When using this operator with a BlockMemoryPool, between 1 and 3 device memory blocks (storage_type = 1) will be required based on the input tensors and parameters:
out_height * out_width * out_channels * out_element_size_bytes where (out_height, out_width) will either be (in_height, in_width) (or (resize_height, resize_width) a resize was specified). out_element_size is the element size in bytes (e.g. 1 for RGB888 or 4 for Float32).resize_height * resize_width * in_channels * in_element_size_bytes.in_height * in_width * in_channels * in_element_size_bytes.Thus when declaring the memory pool, num_blocks should be between 1-3 and block_size should be set to the maximum of the individual blocks sizes described above.
==Notes==
This operator may launch CUDA kernels that execute asynchronously on a CUDA stream. As a result, the compute method may return before all GPU work has completed. Downstream operators that receive data from this operator should either:
op_input.receive_cuda_stream(<port_name>) (after calling receive for that port) to synchronize the CUDA stream with the downstream operator’s dedicated internal stream before accessing the data.CudaStreamCondition to delay scheduling until upstream GPU work has completed.For more details on CUDA stream handling in Holoscan, see the {ref}CUDA Stream Handling in Holoscan Applications <holoscan-cuda-stream-handling> section of the Holoscan SDK documentation.
Inherits from: holoscan::Operator (public)
Define the operator specification.
Parameters
The reference to the operator specification.
Initialize the component.
This method is called only once when the component is created for the first time, and use of light-weight initialization.
Implement the startup logic of the operator.
This method is called multiple times over the lifecycle of the operator according to the order defined in the lifecycle, and used for heavy initialization tasks such as allocating memory resources.
Implement the compute method.
This method is called by the runtime multiple times. The runtime calls this method until the operator is stopped.
Parameters
The input context of the operator.
The output context of the operator.
The execution context of the operator.
Implement the shutdown logic of the operator.
This method is called multiple times over the lifecycle of the operator according to the order defined in the lifecycle, and used for heavy deinitialization tasks such as deallocation of all resources previously assigned in start.
Get the operator type.
Returns: The operator type.
Set the name of the operator.
Returns: The reference to this operator.
Parameters
The name of the operator.
Set the fragment of the operator.
Returns: The reference to this operator.
Parameters
The pointer to the fragment of the operator.
Set the operator spec.
Returns: The reference to this operator.
Parameters
The operator spec.
Get the shared pointer to the operator spec.
Returns: The shared pointer to the operator spec.
Get a shared pointer to the Condition object.
Returns: The reference to the Condition object. If the condition does not exist, return the nullptr.
Parameters
The name of the condition.
Get the conditions of the operator.
Returns: The conditions of the operator.
Get a shared pointer to the Resource object.
Returns: The reference to the Resource object. If the resource does not exist, returns the nullptr.
Parameters
The name of the resource.
Get the resources of the operator.
Returns: The resources of the operator.
Add a condition to the operator.
Parameters
The condition to add.
Returns whether the operator is a root operator based on its fragment’s graph.
Returns: True, if the operator is a root operator; false, otherwise
Returns whether the operator is a user-defined root operator i.e., the first operator added to the graph.
Returns: True, if the operator is a user-defined root operator; false, otherwise
Returns whether the operator is a leaf operator based on its fragment’s graph.
Returns: True, if the operator is a leaf operator; false, otherwise
Returns the fully qualified name of the operator including the name of the fragment.
Returns: std::string fully qualified name of the operator in the format: “<fragment_name>.<operator_name>“
Get a YAML representation of the operator.
Returns: YAML node including type, specs, conditions and resources of the operator in addition to the base component properties.
Get the GXF GraphEntity object corresponding to this operator.
Returns: graph entity corresponding to the operator
Get the current message label for a given input port.
This method retrieves the MessageLabel associated with data received on the specified input port. The MessageLabel contains timing and path information for data flow tracking.
This method should ideally be called after receiving data on the input port.
If the input_port_name is invalid (not found in the operator’s input ports), an error is logged and an empty MessageLabel is returned.
Returns: MessageLabel The current message label for the input port. Returns an empty MessageLabel if the port does not have a message label or if the port does not exist (logs an error in the latter case as well).
Throws: std::runtime_error If the operator backend is not GXF-compatible.
Throws: std::runtime_error If fragment is not set.
Throws: std::runtime_error If operator spec is not set.
Throws: std::runtime_error If fragment flow tracking is not enabled.
Parameters
The name of the input port.
Get a shared pointer to the dynamic metadata of this operator.
Note: currently this metadata dictionary is only active if explicitly enabled for the application by setting Fragment::is_metadata_enabled(true). When metadata is disabled the dictionary will not be populated by receive calls and will not be emitted on emit calls.
This metadata dictionary is always empty at the start of each compute call. It is populated by metadata received on input ports during InputContext::receive() calls and can be modified as desired by the operator during the compute call. Any metadata corresponding to this object will be sent on the output ports by any OutputContext::emit() calls.
Returns: The metadata dictionary for this operator.
Determine if metadata is enabled for this operator.
Returns: Boolean indicating if metadata is enabled (returns fragment()``->``is_metadata_enabled() if enable_metadata was not explicitly called for the operator.
Enable or disable metadata for this operator.
If this method has not been used to explicitly enable or disable metadata, the value for is_metadata_enabled() will be determined by Fragment::is_metadata_enabled() when the operator is initialized.
Parameters
Boolean indicating if metadata should be enabled.
Set the metadata update policy used by this operator.
The metadata policy determines how metadata is merged across multiple receive calls:
MetadataPolicy::kUpdate: Update the existing value when a key already exists.MetadataPolicy::kInplaceUpdate: Update the existing MetadataObject’s value in-place when a key already exists.MetadataPolicy::kReject: Do not modify the existing value if a key already exists.MetadataPolicy::kRaise: Raise an exception if a key already exists (default).Parameters
The metadata update policy to be used by this operator.
If no CudaStreamPool parameter or argument already exists, add a default one.
This method is available to be called by derived classes to add a default CudaStreamPool in This method is available to be called by derived classes to add a default CudaStreamPool inthe case that the user did not pass one in as an argument tomake_operator`.
This function will not add an additional CUDA stream pool if one was already passed in as an argument to make_operator (i.e. it is in resources_) or if a “cuda_stream_pool” parameter already exists in the operator spec.
/** Return the Receiver corresponding to a specific input port.
Returns: The Receiver corresponding to the input port, if it exists. Otherwise, return nullopt.
Parameters
The name of the input port.
Return the Transmitter corresponding to a specific output port.
Returns: The Transmitter corresponding to the output port, if it exists. Otherwise, return nullopt.
Parameters
The name of the output port.
Bind an input port to a Pub/Sub topic.
This is the programmatic override path for topic-mapped ports and takes precedence over any lower-priority IOSpec::topic() default configured in setup().
Parameters
The input port to bind.
The topic name to subscribe to.
Optional QoS profile override. When nullopt, any QoS already configured on the port (e.g. in setup()) is preserved.
If true, replace an explicitly non-PubSub connector on this port with a Pub/Sub connector.
Bind an output port to a Pub/Sub topic.
This is the programmatic override path for topic-mapped ports and takes precedence over any lower-priority IOSpec::topic() default configured in setup().
Parameters
The output port to bind.
The topic name to publish to.
Optional QoS profile override. When nullopt, any QoS already configured on the port (e.g. in setup()) is preserved.
If true, replace an explicitly non-PubSub connector on this port with a Pub/Sub connector.
Get the configured topic name for an input port, if any.
Get the configured topic name for an output port, if any.
Get the effective QoS profile for an input port, if it is topic-mapped.
Get the effective QoS profile for an output port, if it is topic-mapped.
Set the queue policy to be used by an input or output port.
The following IOSpec::QueuePolicy values are supported:
Parameters
The name of the port.
Enum flag indicating whether port_name specifies an input or output port.
The queue policy to set for the port.
Get the list of next flows connected to this operator.
Returns: A vector of FlowInfo objects representing the flows to downstream operators.
Add a dynamic flow from this operator to another operator using a FlowInfo object.
Parameters
The flow information object describing the connection between operators.
Get the list of dynamic flows that have been added to this operator.
Returns: A shared pointer to a vector of FlowInfo objects representing the dynamic flows.
Locate a flow info in the operator’s next flows based on a given predicate.
Returns: Shared pointer to the matching FlowInfo, or nullptr if not found.
Parameters
Lambda function that takes a FlowInfo shared pointer and returns a boolean.
Find all FlowInfo objects in the operator’s next flows that match a given condition.
Returns: A vector of shared pointers to the matching FlowInfo objects.
Parameters
A lambda function that takes a shared pointer to a FlowInfo object and returns a boolean.
Get the internal asynchronous condition for the operator.
Note: This object is only accessible after the executor has called Operator::initialize() via run() or run_async(). If accessed during Application::compose(), it will return nullptr.
Returns: A shared pointer to the internal asynchronous condition.
Stop the execution of the operator.
This method is used to stop the execution of the operator by setting the internal async condition to EVENT_NEVER state, which sets the scheduling condition to NEVER. Once stopped, the operator will not be scheduled for execution (the compute() method will not be called).
Note that executing this method does not trigger the operator’s stop() method. The stop() method is called only when the scheduler deactivates all operators together.
Get the ExecutionContext object.
Returns: The shared pointer to the ExecutionContext object.
Ensure the contexts (input/output/execution) for the operator.
This method is called by the GXFExecutor when the operator is initialized.
Internal method to clean up operator resources and prevent circular references.
This is an internal method called automatically by the GXFWrapper during operator shutdown (in GXFWrapper::stop() method). It resets std::shared_ptr fields and std::function objects such as input_exec_spec_, output_exec_spec_, next_flows_, dynamic_flows_, and dynamic_flow_func_ to break potential circular references between connected Operator objects.
This is an internal method that should never be called directly by user code. Improper use can lead to undefined behavior.
Check if the operator is a GXF compatible operator type.
This checks the OperatorType and returns true if it is one of {kNative, kGXF, kVirtual}.
Returns: True if the operator is a GXF compatible operator type, false otherwise.
Check if the operator has any UCX connectors.
Check if the operator has any PubSub connectors.
Check if the operator has any network (UCX or PubSub) connectors.
Get the list of arguments.
Returns: The vector of arguments.
Get a description of the component.
Returns: YAML string.
See also: to_yaml_node()
Retrieve a registered fragment service or resource.
Retrieves a previously registered fragment service or resource by its type and optional identifier. Returns nullptr if no service/resource is found with the specified type and identifier.
Note that any changes to the service retrieval logic in this method should be synchronized with the implementation in Fragment::service() method to maintain consistency.
Returns: The shared pointer to the service/resource, or nullptr if not found or if type casting fails.
Template parameters
The type of the service/resource to retrieve. Must inherit from either Resource or FragmentService. Defaults to DefaultFragmentService if not specified.
Parameters
The identifier of the service/resource. If empty, retrieves by type only.
Retrieve a registered fragment service or resource for Python bindings.
This is a helper method for Python bindings to retrieve a service by its C++ type info.
Returns: The shared pointer to the base service, or nullptr if not found.
Parameters
The type info of the service/resource to retrieve.
The identifier of the service/resource. If empty, retrieves by type only.
Reset any backend-specific objects (e.g. GXF GraphEntity).
This function creates a GraphEntity corresponding to the operator.
Returns: The GXF entity eid corresponding to the graph entity.
Parameters
The GXF context.
Prefix to add to the operator’s name when creating the GraphEntity.
Initialize the internal asynchronous condition to control the operator execution.
This method is called by the GXFExecutor when the operator is initialized.
Add this operator as the codelet in the GXF GraphEntity.
Returns: The codelet component id corresponding to GXF codelet.
Replace any “receiver” supplied as a string with the actual receiver of that name.
Can only be called after GXFExecutor::create_input_port so the input ports (receivers) exist.
Determine ports whose transmitter or receiver are associated with an Arg for a Condition.
Should be called before GXFExecutor::create_input_port or GXFExecutor::create_output_port.
This function returns a consolidated MessageLabel for all the input ports of an Operator.
If there is no input port (root Operator), then a new MessageLabel with the current Operator and default receive timestamp is returned.
Returns: The consolidated MessageLabel
Update the input_message_labels map with the given MessageLabel a corresponding input_name.
Parameters
The input port name for which the MessageLabel is updated
The new MessageLabel that will be set for the input port
Delete the input_message_labels map entry for the given input_name.
Parameters
The input port name for which the MessageLabel is deleted
Reset the input message labels to clear all its contents.
This is done for a leaf operator when it finishes its execution as it is assumed that all its inputs are processed.
Check if the operator has any input message labels.
Get the number of published messages for each output port indexed by the output port name.
The function is utilized by the DFFTCollector to update the DataFlowTracker with the number of published messages for root operators.
Returns: The map of the number of published messages for every output name.
This function updates the number of published messages for a given output port.
Parameters
The name of the output port
Set the service provider that owns this component.
Returns whether all the successors of an operator are virtual operators.
Returns: true if the operator has all virtual operator successors, false otherwise
Parameters
The shared_ptr to the operator for which the check is to be performed
The graph of operators. fragment()->graph() can usually be used to get this graph.
Returns whether all the predecessors of an operator are virtual operators.
Returns: true if the operator has all virtual operator predecessors, false otherwise
Parameters
The shared_ptr to the operator for which the check is to be performed
The graph of operators. fragment()->graph() can usually be used to get this graph.
Return operator name and port name from a string in the format of “<op_name>[.<port_name>]“.
Return operator name and port name from a string in the format of “<op_name>-<port_name>“.
Register the codec for serialization/deserialization of a custom type.
Use holoscan::gxf::GXFExecutor::register_codec instead.
Validate operator name against restricted keywords and patterns.
Throws: std::invalid_argument if the name contains restricted patterns.
Parameters
The name to validate.
Register the argument setter for the given type.
If an operator or resource has an argument with a custom type, the argument setter must be registered using this method.
The argument setter is used to set the value of the argument from the YAML configuration.
This method can be called in the initialization phase of the operator/resource (e.g., initialize()). The example below shows how to register the argument setter for the custom type (Vec3):
It is assumed that YAML::convert<T>::encode and YAML::convert<T>::decode are implemented for the given type. You need to specialize the YAML::convert<> template class.
For example, suppose that you had a Vec3 class with the following members:
You can define the YAML::convert<Vec3> as follows in a ‘.cpp’ file:
Please refer to the yaml-cpp documentation for more details.
Template parameters
The type of the argument to register.
Example
Example
Example
Register the argument setter for the given type.
Please refer to the documentation of register_converter() for more details.
Template parameters
The type of the argument to register.
Operator type used by the executor.