holoscan::ThreadPool

Beta

Thread pool resource.

This is a thread pool for use with the EventBasedScheduler or MultiThreadScheduler. This resource should be created via the Fragment::make_thread_pool method instead of the usual Fragment::make_resource method as it requires additional configuration of an associated GXF EntityGroup.

User-defined thread pools create additional worker threads beyond the default thread pool that is automatically created by the scheduler (based on the scheduler’s worker_thread_number parameter). Operators not assigned to a user-defined pool will use the default pool.

pool1 = make_thread_pool(“pool1”, Arg(“initial_size”, static_cast<int64_t>(2)));

The operators can be added via the add method. For strict thread pinning, the pin_operator argument should be true and the initial_size of the thread pool should be at least as large as the number of operators that will be pinned to threads. The optional pin_cores parameter allows restricting the thread to specific CPU cores (supported by EventBasedScheduler only).

pool1.add(op1, true); pool1.add(op2, true, {0, 1}); // Pin to cores 0 and 1

This add method takes care of adding any needed holoscan::CPUThread resource to the operator.

The MultiThreadScheduler’s strict_job_thread_pinning argument can be set true to disallow execution of any other entities on the pinned thread. The EventBasedScheduler always uses strict thread pinning.

#include <holoscan/system_resources.hpp>

Inherits from: holoscan::gxf::GXFSystemResourceBase (public)

Constructors

ThreadPool

Move

Default

Copy

inlineexplicit

template <typename ArgT,
          typename... ArgsT,
          typename = std::enable_if_t<!std::is_base_of_v<::holoscan::Resource, std::decay_t<ArgT>> &&   (std::is_same_v<::holoscan::Arg, std::decay_t<ArgT>> ||   std::is_same_v<::holoscan::ArgList, std::decay_t<ArgT>>)>>
holoscan::ThreadPool::ThreadPool(
    ArgT &&arg,
    ArgsT &&... args
)

Methods

gxf_typename

const char * holoscan::ThreadPool::gxf_typename() const override

The underlying GXF component’s name.

setup

void holoscan::ThreadPool::setup(
    ComponentSpec &spec
) override

Define the resource specification.

Parameters

spec

ComponentSpec &

The reference to the component specification.

size

int64_t holoscan::ThreadPool::size() const

The number of threads currently in the thread pool.

add

Add an operator to the thread pool

Add multiple operators to the thread pool

void holoscan::ThreadPool::add(
    const std::shared_ptr<Operator> &op,
    bool pin_operator = true,
    std::vector<uint32_t> pin_cores = std::vector<uint32_t>()
)

Add an operator to the thread pool.

Parameters

const std::shared_ptr<Operator> &

The operator to add.

pin_operator

boolDefaults to true

Whether the operator should be pinned to a specific thread in the pool.

pin_cores

std::vector<uint32_t>Defaults to std::vector< uint32_t >()

CPU core IDs to pin the worker threads to (empty means no core pinning).

add_realtime

void holoscan::ThreadPool::add_realtime(
    const std::shared_ptr<Operator> &op,
    SchedulingPolicy sched_policy,
    bool pin_operator = true,
    std::vector<uint32_t> pin_cores = std::vector<uint32_t>(),
    uint32_t sched_priority = 0,
    uint64_t sched_runtime = 0,
    uint64_t sched_deadline = 0,
    uint64_t sched_period = 0
)

Add an operator to the thread pool with real-time scheduling capabilities.

Parameters

const std::shared_ptr<Operator> &

The operator to add.

sched_policy

SchedulingPolicy

Real-time scheduling policy (kFirstInFirstOut, kRoundRobin, kDeadline).

pin_operator

boolDefaults to true

Whether the operator should be pinned to a specific thread in the pool.

pin_cores

std::vector<uint32_t>Defaults to std::vector< uint32_t >()

CPU core IDs to pin the worker thread to (empty means no core pinning).

sched_priority

uint32_tDefaults to 0

Thread priority for FirstInFirstOut and RoundRobin policies.

sched_runtime

uint64_tDefaults to 0

Expected worst case execution time in nanoseconds for Deadline policy.

sched_deadline

uint64_tDefaults to 0

Relative deadline in nanoseconds for Deadline policy.

sched_period

uint64_tDefaults to 0

Period in nanoseconds for Deadline policy.

entity_group

Mutable

Const

void holoscan::ThreadPool::entity_group(
    const std::shared_ptr<gxf::EntityGroup> &entity_group
)

Set the entity group for this thread pool.

operators

std::vector<std::shared_ptr<Operator>> holoscan::ThreadPool::operators() const

The operators associated with this thread pool.

to_yaml_node

YAML::Node holoscan::ThreadPool::to_yaml_node() const override

Get a YAML representation of the thread pool.

Returns: YAML node including properties of the base resource and the operators in the pool.

initialize

void holoscan::gxf::GXFResource::initialize() override

Initialize the component.

This method is called only once when the component is created for the first time, and use of light-weight initialization.

resource_type

ResourceType holoscan::Resource::resource_type() const

Get the resource type.

Returns: The resource type.

name

Set the name of the resource (1)

Set the name of the resource (2)

Const

Resource & holoscan::Resource::name(
    const std::string &name
) &

Set the name of the resource.

Returns: The reference to the resource.

Parameters

name

const std::string &

The name of the resource.

fragment

Set the fragment of the resource

Get a pointer to Fragment object

Const

Resource & holoscan::Resource::fragment(
    Fragment *fragment
)

Set the fragment of the resource.

Returns: The reference to the resource.

Parameters

fragment

Fragment *

The pointer to the fragment of the resource.

spec

Set the component specification to the resource

Get the component specification of the resource

Resource & holoscan::Resource::spec(
    const std::shared_ptr<ComponentSpec> &spec
)

Set the component specification to the resource.

Returns: The reference to the resource.

Parameters

spec

const std::shared_ptr<ComponentSpec> &

The component specification.

spec_shared

std::shared_ptr<ComponentSpec> holoscan::Resource::spec_shared()

Get the shared pointer to the component spec.

Returns: The shared pointer to the component spec.

id

int64_t holoscan::ComponentBase::id() const

Get the identifier of the component.

By default, the identifier is set to -1. It is set to a valid value when the component is initialized.

With the default executor (GXFExecutor), the identifier is set to the GXF component ID.

Returns: The identifier of the component.

add_arg

Add an argument to the component (1)

Add an argument to the component (2)

Add a list of arguments to the component (1)

Add a list of arguments to the component (2)

void holoscan::ComponentBase::add_arg(
    const Arg &arg
)

Add an argument to the component.

Parameters

arg

const Arg &

The argument to add.

args

std::vector<Arg> & holoscan::ComponentBase::args()

Get the list of arguments.

Returns: The vector of arguments.

description

std::string holoscan::ComponentBase::description() const

Get a description of the component.

Returns: YAML string.

See also: to_yaml_node()

service

template <typename ServiceT = DefaultFragmentService>
std::shared_ptr<ServiceT> holoscan::ComponentBase::service(
    std::string_view id = ""
) const

Retrieve a registered fragment service or resource.

Retrieves a previously registered fragment service or resource by its type and optional identifier. Returns nullptr if no service/resource is found with the specified type and identifier.

Note that any changes to the service retrieval logic in this method should be synchronized with the implementation in Fragment::service() method to maintain consistency.

Returns: The shared pointer to the service/resource, or nullptr if not found or if type casting fails.

Template parameters

ServiceT

typename

The type of the service/resource to retrieve. Must inherit from either Resource or FragmentService. Defaults to DefaultFragmentService if not specified.

Parameters

std::string_viewDefaults to ""

The identifier of the service/resource. If empty, retrieves by type only.

get_service_by_type_info

std::shared_ptr<FragmentService> holoscan::ComponentBase::get_service_by_type_info(
    const std::type_info &service_type,
    std::string_view id = ""
) const

Retrieve a registered fragment service or resource for Python bindings.

This is a helper method for Python bindings to retrieve a service by its C++ type info.

Returns: The shared pointer to the base service, or nullptr if not found.

Parameters

service_type

const std::type_info &

The type info of the service/resource to retrieve.

std::string_viewDefaults to ""

The identifier of the service/resource. If empty, retrieves by type only.

gxf_context

Mutable

Const

void holoscan::gxf::GXFComponent::gxf_context(
    gxf_context_t gxf_context
)

gxf_eid

Mutable

Const

void holoscan::gxf::GXFComponent::gxf_eid(
    gxf_uid_t gxf_eid
)

gxf_tid

Mutable

Const

void holoscan::gxf::GXFComponent::gxf_tid(
    gxf_tid_t gxf_tid
)

gxf_cid

Mutable

Const

void holoscan::gxf::GXFComponent::gxf_cid(
    gxf_uid_t gxf_cid
)

gxf_cname

Overload 1

Overload 2

std::string & holoscan::gxf::GXFComponent::gxf_cname()

gxf_graph_entity

Overload 1

Overload 2

std::shared_ptr<nvidia::gxf::GraphEntity> holoscan::gxf::GXFComponent::gxf_graph_entity()

gxf_entity_group_name

std::string holoscan::gxf::GXFComponent::gxf_entity_group_name()

The name of the entity group this component belongs to.

gxf_entity_group_id

gxf_uid_t holoscan::gxf::GXFComponent::gxf_entity_group_id()

The group id of the entity group this component belongs to.

gxf_cptr

void * holoscan::gxf::GXFComponent::gxf_cptr()

gxf_component

nvidia::gxf::Handle<nvidia::gxf::Component> holoscan::gxf::GXFComponent::gxf_component()

gxf_initialize

void holoscan::gxf::GXFComponent::gxf_initialize()

set_gxf_parameter

void holoscan::gxf::GXFComponent::set_gxf_parameter(
    const std::string &component_name,
    const std::string &key,
    ParameterWrapper &param_wrap
)

Set a given parameter on the underlying GXF component.

reset_gxf_graph_entity

void holoscan::gxf::GXFComponent::reset_gxf_graph_entity()

add_to_graph_entity

Overload 1

Overload 2

virtual

virtual void holoscan::gxf::GXFResource::add_to_graph_entity(
    Operator *op
)

set_parameters

void holoscan::gxf::GXFResource::set_parameters() override

This method is invoked by GXFResource::initialize().

By overriding this method, we can modify how GXF Codelet’s parameters are set from the arguments.

handle_dev_id

bool holoscan::gxf::GXFResource::handle_dev_id(
    std::optional<int32_t> &dev_id_value
)

reset_backend_objects

void holoscan::gxf::GXFResource::reset_backend_objects() override

Reset any backend-specific state.

update_params_from_args

Update parameters based on the specified arguments

Update parameters based on the specified arguments (with params)

void holoscan::Resource::update_params_from_args()

Update parameters based on the specified arguments.

service_provider

void holoscan::ComponentBase::service_provider(
    FragmentServiceProvider *provider
)

Set the service provider that owns this component.

Static methods

register_converter

template <typename typeT>
static void holoscan::ComponentBase::register_converter()

If an operator or resource has an argument with a custom type, the argument setter must be registered using this method.

The argument setter is used to set the value of the argument from the YAML configuration.

This method can be called in the initialization phase of the operator/resource (e.g., initialize()). The example below shows how to register the argument setter for the custom type (Vec3):

It is assumed that YAML::convert<T>::encode and YAML::convert<T>::decode are implemented for the given type. You need to specialize the YAML::convert<> template class.

For example, suppose that you had a Vec3 class with the following members:

You can define the YAML::convert<Vec3> as follows in a ‘.cpp’ file:

Please refer to the yaml-cpp documentation for more details.

Template parameters

typeT

typename

The type of the argument to register.

Example

void MyOp::initialize() {
  register_converter<Vec3>();
}

Example

struct Vec3 {
  // make sure you have overloaded operator==() for the comparison
  double x, y, z;
};

Example

namespace YAML {
template<>
struct convert<Vec3> {
  static Node encode(const Vec3& rhs) {
    Node node;
    node.push_back(rhs.x);
    node.push_back(rhs.y);
    node.push_back(rhs.z);
    return node;
  }
  static bool decode(const Node& node, Vec3& rhs) {
    if(!node.IsSequence() || node.size() != 3) {
      return false;
    }
    rhs.x = node[0].as<double>();
    rhs.y = node[1].as<double>();
    rhs.z = node[2].as<double>();
    return true;
  }
};
}

register_argument_setter

template <typename typeT>
void holoscan::ComponentBase::register_argument_setter()

Please refer to the documentation of register_converter() for more details.

Template parameters

typeT

typename

The type of the argument to register.

Types

ResourceType

Resource type used for the initialization of the resource.

Name	Value	Description
`kNative`		Native resource.
`kGXF`		GXF resource.

Member variables

Name	Type	Description
`initial_size_`	`Parameter< int64_t >`	Initial size of the thread pool.
`entity_group_`	`std::shared_ptr< gxf::EntityGroup >`	The entity group associated with the thread pool.
`operators_`	`std::vector< std::shared_ptr< Operator > >`	The operators associated with the thread pool.
`gxf_typename_`	`std::string`	The GXF type name (used for GXFComponentResource).
`resource_type_`	`ResourceType`	The type of the resource.
`is_initialized_`	`bool`	Whether the resource is initialized.
`spec_`	`std::shared_ptr< ComponentSpec >`	The component specification.
`id_`	`int64_t`	The ID of the component.
`name_`	`std::string`	Name of the component.
`fragment_`	`Fragment *`	Pointer to the fragment that owns this component.
`args_`	`std::vector< Arg >`	List of arguments.
`service_provider_`	`FragmentServiceProvider *`	Pointer to the service provider.
`gxf_context_`	`gxf_context_t`
`gxf_eid_`	`gxf_uid_t`
`gxf_tid_`	`gxf_tid_t`
`gxf_cid_`	`gxf_uid_t`
`gxf_graph_entity_`	`std::shared_ptr< nvidia::gxf::GraphEntity >`
`gxf_cname_`	`std::string`
`gxf_component_`	`nvidia::gxf::Handle< nvidia::gxf::Component >`
`gxf_cptr_`	`void *`