> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/holoscan/sdk-user-guide/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sdk-user-guide/_mcp/server.

# holoscan::StreamOrderedAllocator

> CUDA device memory allocator using stream-ordered allocation.

CUDA device memory allocator using stream-ordered allocation.

`StreamOrderedAllocator` uses CUDA's stream-ordered memory allocator (`cudaMallocAsync`/`cudaFreeAsync`) to dynamically allocate device memory. Stream-ordered allocation enables memory operations to be tied to specific CUDA streams, allowing allocation and deallocation without blocking the host or other streams.

See the CUDA Programming Guide section on [Stream-Ordered Memory Allocator](https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/stream-ordered-memory-allocation.html#stream-ordered-memory-allocator) for details on the underlying CUDA feature.

This allocator only supports CUDA device memory. If host memory is also needed, see [`RMMAllocator`](rmmallocator) which provides both device and pinned host memory pools.

Because it is a [CudaAllocator](cudaallocator) it supports both synchronous ([`allocate`](allocator#allocate), [`free`](allocator#free)) and asynchronous ([`allocate_async`](cudaallocator#allocateasync), [`free_async`](cudaallocator#freeasync)) APIs for memory allocation.

The values for the memory parameters, such as `device_memory_initial_size` must be specified in the form of a string containing a non-negative integer value followed by a suffix representing the units. Supported units are B, KB, MB, GB and TB where the values are powers of 1024 bytes (e.g. MB = 1024 \* 1024 bytes). Examples of valid units are "512MB", "256 KB", "1 GB". If a floating point number is specified that decimal portion will be truncated (i.e. the value is rounded down to the nearest integer).

\==Parameters==

* **device\_memory\_initial\_size** (std::string, optional): The initial size of the device memory pool. See above for the format accepted. Defaults to "8MB" on aarch64 and "16MB" on x86\_64.
* **device\_memory\_max\_size** (std::string, optional): The maximum size of the device memory pool. See above for the format accepted. The default is to use twice the value set for `device_memory_initial_size`.
* **release\_threshold** (std::string, optional): The amount of reserved memory to hold onto before trying to release memory back to the OS. See above for the format accepted. The default value is "4MB".
* **dev\_id** (int32\_t, optional): The CUDA device id specifying which device the memory pool will use. (Default: 0)

```cpp showLineNumbers={false}
#include <holoscan/stream_ordered_allocator.hpp>
```

**Inherits from:** `holoscan::CudaAllocator` (public)

***

## Constructors

### StreamOrderedAllocator \[#streamorderedallocator]

inline

explicit

```cpp showLineNumbers={false}
template <typename ArgT,
          typename... ArgsT,
          typename = std::enable_if_t<!std::is_base_of_v<::holoscan::Resource, std::decay_t<ArgT>> &&   (std::is_same_v<::holoscan::Arg, std::decay_t<ArgT>> ||   std::is_same_v<::holoscan::ArgList, std::decay_t<ArgT>>)>>
holoscan::StreamOrderedAllocator::StreamOrderedAllocator(
    ArgT &&arg,
    ArgsT &&... args
)
```

```cpp showLineNumbers={false}
holoscan::StreamOrderedAllocator::StreamOrderedAllocator() = default
```

```cpp showLineNumbers={false}
holoscan::StreamOrderedAllocator::StreamOrderedAllocator(
    const std::string &name,
    nvidia::gxf::StreamOrderedAllocator *component
)
```

***

## Methods

### gxf\_typename \[#gxftypename]

```cpp showLineNumbers={false}
const char * holoscan::StreamOrderedAllocator::gxf_typename() const override
```

### setup \[#setup]

```cpp showLineNumbers={false}
void holoscan::StreamOrderedAllocator::setup(
    ComponentSpec &spec
) override
```

Define the resource specification.

**Parameters**

The reference to the component specification.

### get \[#get]

```cpp showLineNumbers={false}
nvidia::gxf::StreamOrderedAllocator * holoscan::StreamOrderedAllocator::get() const
```

### allocate\_async \[#allocateasync]

```cpp showLineNumbers={false}
nvidia::byte * holoscan::StreamOrderedAllocator::allocate_async(
    uint64_t size,
    cudaStream_t stream
)
```

### free\_async \[#freeasync]

```cpp showLineNumbers={false}
void holoscan::StreamOrderedAllocator::free_async(
    byte *pointer,
    cudaStream_t stream
)
```

### pool\_size \[#poolsize]

```cpp showLineNumbers={false}
size_t holoscan::StreamOrderedAllocator::pool_size(
    MemoryStorageType type
) const
```

### is\_available \[#isavailable]

```cpp showLineNumbers={false}
virtual bool holoscan::Allocator::is_available(
    uint64_t size
)
```

### allocate \[#allocate]

```cpp showLineNumbers={false}
virtual nvidia::byte * holoscan::Allocator::allocate(
    uint64_t size,
    MemoryStorageType type
)
```

### free \[#free]

```cpp showLineNumbers={false}
virtual void holoscan::Allocator::free(
    nvidia::byte *pointer
)
```

### block\_size \[#blocksize]

```cpp showLineNumbers={false}
uint64_t holoscan::Allocator::block_size()
```

### initialize \[#initialize]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFResource::initialize() override
```

Initialize the component.

This method is called only once when the component is created for the first time, and use of light-weight initialization.

### resource\_type \[#resourcetype]

```cpp showLineNumbers={false}
ResourceType holoscan::Resource::resource_type() const
```

Get the resource type.

**Returns:** The resource type.

### name \[#name]

```cpp showLineNumbers={false}
Resource & holoscan::Resource::name(
    const std::string &name
) &
```

Set the name of the resource.

**Returns:** The reference to the resource.

**Parameters**

The name of the resource.

```cpp showLineNumbers={false}
Resource && holoscan::Resource::name(
    const std::string &name
) &&
```

Set the name of the resource.

**Returns:** The reference to the resource.

**Parameters**

The name of the resource.

const

```cpp showLineNumbers={false}
const std::string & holoscan::ComponentBase::name() const
```

Get the name of the component.

**Returns:** The name of the component.

### fragment \[#fragment]

```cpp showLineNumbers={false}
Resource & holoscan::Resource::fragment(
    Fragment *fragment
)
```

Set the fragment of the resource.

**Returns:** The reference to the resource.

**Parameters**

The pointer to the fragment of the resource.

```cpp showLineNumbers={false}
Fragment * holoscan::ComponentBase::fragment()
```

Get a pointer to [Fragment](fragment) object.

**Returns:** The Pointer to [Fragment](fragment) object.

const

```cpp showLineNumbers={false}
const Fragment * holoscan::ComponentBase::fragment() const
```

Get a const pointer to [Fragment](fragment) object.

**Returns:** The const pointer to [Fragment](fragment) object.

### spec \[#spec]

```cpp showLineNumbers={false}
Resource & holoscan::Resource::spec(
    const std::shared_ptr<ComponentSpec> &spec
)
```

Set the component specification to the resource.

**Returns:** The reference to the resource.

**Parameters**

The component specification.

```cpp showLineNumbers={false}
ComponentSpec * holoscan::Resource::spec()
```

Get the component specification of the resource.

**Returns:** The pointer to the component specification.

### spec\_shared \[#specshared]

```cpp showLineNumbers={false}
std::shared_ptr<ComponentSpec> holoscan::Resource::spec_shared()
```

Get the shared pointer to the component spec.

**Returns:** The shared pointer to the component spec.

### id \[#id]

```cpp showLineNumbers={false}
int64_t holoscan::ComponentBase::id() const
```

Get the identifier of the component.

By default, the identifier is set to -1. It is set to a valid value when the component is initialized.

With the default executor (GXFExecutor), the identifier is set to the GXF component ID.

**Returns:** The identifier of the component.

### add\_arg \[#addarg]

```cpp showLineNumbers={false}
void holoscan::ComponentBase::add_arg(
    const Arg &arg
)
```

Add an argument to the component.

**Parameters**

The argument to add.

```cpp showLineNumbers={false}
void holoscan::ComponentBase::add_arg(
    Arg &&arg
)
```

Add an argument to the component.

**Parameters**

The argument to add.

```cpp showLineNumbers={false}
void holoscan::ComponentBase::add_arg(
    const ArgList &arg
)
```

Add a list of arguments to the component.

**Parameters**

The list of arguments to add.

```cpp showLineNumbers={false}
void holoscan::ComponentBase::add_arg(
    ArgList &&arg
)
```

Add a list of arguments to the component.

**Parameters**

The list of arguments to add.

### args \[#args]

```cpp showLineNumbers={false}
std::vector<Arg> & holoscan::ComponentBase::args()
```

Get the list of arguments.

**Returns:** The vector of arguments.

### description \[#description]

```cpp showLineNumbers={false}
std::string holoscan::ComponentBase::description() const
```

Get a description of the component.

**Returns:** `YAML` string.

**See also:**
to\_yaml\_node()

### service \[#service]

```cpp showLineNumbers={false}
template <typename ServiceT = DefaultFragmentService>
std::shared_ptr<ServiceT> holoscan::ComponentBase::service(
    std::string_view id = ""
) const
```

Retrieve a registered fragment service or resource.

Retrieves a previously registered fragment service or resource by its type and optional identifier. Returns nullptr if no service/resource is found with the specified type and identifier.

Note that any changes to the service retrieval logic in this method should be synchronized with the implementation in `Fragment::service()` method to maintain consistency.

**Returns:** The shared pointer to the service/resource, or nullptr if not found or if type casting fails.

**Template parameters**

The type of the service/resource to retrieve. Must inherit from either [Resource](resource) or [FragmentService](fragmentservice). Defaults to [DefaultFragmentService](defaultfragmentservice) if not specified.

**Parameters**

The identifier of the service/resource. If empty, retrieves by type only.

### get\_service\_by\_type\_info \[#getservicebytypeinfo]

```cpp showLineNumbers={false}
std::shared_ptr<FragmentService> holoscan::ComponentBase::get_service_by_type_info(
    const std::type_info &service_type,
    std::string_view id = ""
) const
```

Retrieve a registered fragment service or resource for Python bindings.

This is a helper method for Python bindings to retrieve a service by its C++ type info.

**Returns:** The shared pointer to the base service, or nullptr if not found.

**Parameters**

The type info of the service/resource to retrieve.

The identifier of the service/resource. If empty, retrieves by type only.

### gxf\_context \[#gxfcontext]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_context(
    gxf_context_t gxf_context
)
```

const

```cpp showLineNumbers={false}
gxf_context_t holoscan::gxf::GXFComponent::gxf_context() const
```

### gxf\_eid \[#gxfeid]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_eid(
    gxf_uid_t gxf_eid
)
```

const

```cpp showLineNumbers={false}
gxf_uid_t holoscan::gxf::GXFComponent::gxf_eid() const
```

### gxf\_tid \[#gxftid]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_tid(
    gxf_tid_t gxf_tid
)
```

const

```cpp showLineNumbers={false}
gxf_tid_t holoscan::gxf::GXFComponent::gxf_tid() const
```

### gxf\_cid \[#gxfcid]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_cid(
    gxf_uid_t gxf_cid
)
```

const

```cpp showLineNumbers={false}
gxf_uid_t holoscan::gxf::GXFComponent::gxf_cid() const
```

### gxf\_cname \[#gxfcname]

```cpp showLineNumbers={false}
std::string & holoscan::gxf::GXFComponent::gxf_cname()
```

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_cname(
    const std::string &name
)
```

### gxf\_graph\_entity \[#gxfgraphentity]

```cpp showLineNumbers={false}
std::shared_ptr<nvidia::gxf::GraphEntity> holoscan::gxf::GXFComponent::gxf_graph_entity()
```

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_graph_entity(
    std::shared_ptr<nvidia::gxf::GraphEntity> graph_entity
)
```

### gxf\_entity\_group\_name \[#gxfentitygroupname]

```cpp showLineNumbers={false}
std::string holoscan::gxf::GXFComponent::gxf_entity_group_name()
```

The name of the entity group this component belongs to.

### gxf\_entity\_group\_id \[#gxfentitygroupid]

```cpp showLineNumbers={false}
gxf_uid_t holoscan::gxf::GXFComponent::gxf_entity_group_id()
```

The group id of the entity group this component belongs to.

### gxf\_cptr \[#gxfcptr]

```cpp showLineNumbers={false}
void * holoscan::gxf::GXFComponent::gxf_cptr()
```

### gxf\_component \[#gxfcomponent]

```cpp showLineNumbers={false}
nvidia::gxf::Handle<nvidia::gxf::Component> holoscan::gxf::GXFComponent::gxf_component()
```

### gxf\_initialize \[#gxfinitialize]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::gxf_initialize()
```

### set\_gxf\_parameter \[#setgxfparameter]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::set_gxf_parameter(
    const std::string &component_name,
    const std::string &key,
    ParameterWrapper &param_wrap
)
```

Set a given parameter on the underlying GXF component.

### reset\_gxf\_graph\_entity \[#resetgxfgraphentity]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFComponent::reset_gxf_graph_entity()
```

### add\_to\_graph\_entity \[#addtographentity]

virtual

```cpp showLineNumbers={false}
virtual void holoscan::gxf::GXFResource::add_to_graph_entity(
    Operator *op
)
```

```cpp showLineNumbers={false}
void holoscan::gxf::GXFResource::add_to_graph_entity(
    Fragment *fragment,
    const std::shared_ptr<nvidia::gxf::GraphEntity> &graph_entity
)
```

### to\_yaml\_node \[#toyamlnode]

```cpp showLineNumbers={false}
YAML::Node holoscan::gxf::GXFResource::to_yaml_node() const override
```

Get a `YAML` representation of the resource.

**Returns:** `YAML` node including type and specs of the resource in addition to the base component properties.

### set\_parameters \[#setparameters]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFResource::set_parameters() override
```

This method is invoked by `GXFResource::initialize()`.

By overriding this method, we can modify how GXF Codelet's parameters are set from the arguments.

### handle\_dev\_id \[#handledevid]

```cpp showLineNumbers={false}
bool holoscan::gxf::GXFResource::handle_dev_id(
    std::optional<int32_t> &dev_id_value
)
```

### reset\_backend\_objects \[#resetbackendobjects]

```cpp showLineNumbers={false}
void holoscan::gxf::GXFResource::reset_backend_objects() override
```

Reset any backend-specific state.

### update\_params\_from\_args \[#updateparamsfromargs]

```cpp showLineNumbers={false}
void holoscan::Resource::update_params_from_args()
```

Update parameters based on the specified arguments.

```cpp showLineNumbers={false}
void holoscan::ComponentBase::update_params_from_args(
    std::unordered_map<std::string, ParameterWrapper> &params
)
```

Update parameters based on the specified arguments.

### service\_provider \[#serviceprovider]

```cpp showLineNumbers={false}
void holoscan::ComponentBase::service_provider(
    FragmentServiceProvider *provider
)
```

Set the service provider that owns this component.

***

## Static methods

### register\_converter \[#registerconverter]

```cpp showLineNumbers={false}
template <typename typeT>
static void holoscan::ComponentBase::register_converter()
```

Register the argument setter for the given type.

If an operator or resource has an argument with a custom type, the argument setter must be registered using this method.

The argument setter is used to set the value of the argument from the `YAML` configuration.

This method can be called in the initialization phase of the operator/resource (e.g., `initialize()`). The example below shows how to register the argument setter for the custom type (`Vec3`):

It is assumed that `YAML::convert<T>::encode` and `YAML::convert<T>::decode` are implemented for the given type. You need to specialize the `YAML::convert<>` template class.

For example, suppose that you had a `Vec3` class with the following members:

You can define the `YAML::convert<Vec3>` as follows in a '.cpp' file:

Please refer to the [yaml-cpp documentation](https://github.com/jbeder/yaml-cpp/wiki/Tutorial#converting-tofrom-native-data-types) for more details.

**Template parameters**

The type of the argument to register.

**Example**

```cpp showLineNumbers={false}
void MyOp::initialize() {
  register_converter<Vec3>();
}
```

**Example**

```cpp showLineNumbers={false}
struct Vec3 {
  // make sure you have overloaded operator==() for the comparison
  double x, y, z;
};
```

**Example**

```cpp showLineNumbers={false}
namespace YAML {
template<>
struct convert<Vec3> {
  static Node encode(const Vec3& rhs) {
    Node node;
    node.push_back(rhs.x);
    node.push_back(rhs.y);
    node.push_back(rhs.z);
    return node;
  }

  static bool decode(const Node& node, Vec3& rhs) {
    if(!node.IsSequence() || node.size() != 3) {
      return false;
    }

    rhs.x = node[0].as<double>();
    rhs.y = node[1].as<double>();
    rhs.z = node[2].as<double>();
    return true;
  }
};
}
```

### register\_argument\_setter \[#registerargumentsetter]

```cpp showLineNumbers={false}
template <typename typeT>
void holoscan::ComponentBase::register_argument_setter()
```

Register the argument setter for the given type.

Please refer to the documentation of `register_converter()` for more details.

**Template parameters**

The type of the argument to register.

***

## Types

### ResourceType

[Resource](resource) type used for the initialization of the resource.

| Name      | Value | Description      |
| --------- | ----- | ---------------- |
| `kNative` |       | Native resource. |
| `kGXF`    |       | GXF resource.    |

***

## Member variables

| Name                          | Type                                            | Description                                                                |
| ----------------------------- | ----------------------------------------------- | -------------------------------------------------------------------------- |
| `release_threshold_`          | `Parameter< std::string >`                      |                                                                            |
| `device_memory_initial_size_` | `Parameter< std::string >`                      |                                                                            |
| `device_memory_max_size_`     | `Parameter< std::string >`                      |                                                                            |
| `dev_id_`                     | `Parameter< int32_t >`                          |                                                                            |
| `gxf_typename_`               | `std::string`                                   | The GXF type name (used for [GXFComponentResource](gxfcomponentresource)). |
| `resource_type_`              | `ResourceType`                                  | The type of the resource.                                                  |
| `is_initialized_`             | `bool`                                          | Whether the resource is initialized.                                       |
| `spec_`                       | `std::shared_ptr< ComponentSpec >`              | The component specification.                                               |
| `id_`                         | `int64_t`                                       | The ID of the component.                                                   |
| `name_`                       | `std::string`                                   | Name of the component.                                                     |
| `fragment_`                   | `Fragment *`                                    | Pointer to the fragment that owns this component.                          |
| `args_`                       | `std::vector< Arg >`                            | List of arguments.                                                         |
| `service_provider_`           | `FragmentServiceProvider *`                     | Pointer to the service provider.                                           |
| `gxf_context_`                | `gxf_context_t`                                 |                                                                            |
| `gxf_eid_`                    | `gxf_uid_t`                                     |                                                                            |
| `gxf_tid_`                    | `gxf_tid_t`                                     |                                                                            |
| `gxf_cid_`                    | `gxf_uid_t`                                     |                                                                            |
| `gxf_graph_entity_`           | `std::shared_ptr< nvidia::gxf::GraphEntity >`   |                                                                            |
| `gxf_cname_`                  | `std::string`                                   |                                                                            |
| `gxf_component_`              | `nvidia::gxf::Handle< nvidia::gxf::Component >` |                                                                            |
| `gxf_cptr_`                   | `void *`                                        |                                                                            |