Clara Holoscan Deploy 0.7.4
0.7.4

7.6. Clara Pipeline Fast I/O Context

While the nvidia_clara_payload traditionally provides disk-based input and output support to applications, it is often the case that medical imaging pipelines will be operating on very large data sets, sharing and/or passing data between the operators in the pipeline, and swapping this data to and from disk between each operator is inefficient in both runtime and resource usage.

To avoid the inefficient use of disk-based data passing, the nvidia_clara_payload also provides a Fast I/O Context, nvidia_clara_fastio_context, which is an interface that provides access to memory resources that are accessible by all operators running within the same pipeline, and which can be used for for efficient, zero-copy sharing and passing of data between operators.

The Fast I/O Context provides not just the means to allocate Fast I/O memory resources (nvidia_clara_fastio__create), but also provides a mechanism to pass these resources between multiple operators in a pipeline using basic string identifiers (using nvidia_clara_fastio__publish and nvidia_clara_fastio__get)

An nvidia_clara_fastio_context pointer can be retrieved by calling nvidia_clara_payload__fastio_context with the nvidia_clara_payload pointer provided to the application as part of the Clara Pipeline Operator Life-Cycle execution event callback. Application code should not free this nvidia_clara_fastio_context, nor should it retain the pointer beyond the lifetime of the application supplied execution callback handler.

see also: Fast I/O Allocations, Payload, and Driver Callbacks

Allocates a Fast I/O memory resource of the requested size. Returns an nvidia_clara_fastio_alloc handle to identify the allocation.

Allocations returned by this function are initially only accessible by the calling process using the allocation handle that was returned, and it’s the caller’s responsibility to call one of nvidia_clara_fastio__release or nvidia_clara_fastio__free in order to release any local references to the resource before returning from the execute callback. Failing to do so may lead to allocations persisting beyond the required lifespan, leading to wasted and/or insufficient resource issues during pipeline execution.

In order to make a Fast I/O allocation available to other operators in the pipeline, the allocating process must publish the allocation to the Fast I/O context using nvidia_clara_fastio__publish, after which other operators may access and import a local reference to that same resource using nvidia_clara_fastio__get.

see also: Fast I/O Allocations

Parameters

  • context

    type: nvidia_clara_fastio_context *

    direction: In

    Pointer to the Fast I/O Context.

  • size

    type: size_t

    direction: In

    Size, in bytes, of the requested allocation.

  • alloc_out

    type: nvidia_clara_fastio_alloc **

    direction: Out

    Pointer to the allocated Fast I/O allocation.

Return

Returns zero if successful; otherwise a non-zero value.

Copy
Copied!
            

result nvidia_clara_fastio__create( IN nvidia_clara_fastio_context *context, IN size_t size, OUT nvidia_clara_fastio_alloc **alloc_out );

Each Fast I/O allocation begins as a single nvidia_clara_fastio_alloc handle that is returned to the operator that created it using nvidia_clara_fastio__create. Fast I/O allocations are not accessible to other operators until nvidia_clara_fastio__publish is used to explicitly publish the allocation.

While nvidia_clara_fastio_alloc objects act as local references to their underlying Fast I/O memory resources, they also store some basic metadata state that can be used to help describe the resources. For example, nvidia_clara_fastio__set_type_and_dimensions can be used to set the element type and array dimensions for Fast I/O allocations being used to store typed, N-dimensional arrays.

When multiple operators share access to a single Fast I/O memory resource, each operator will create its own nvidia_clara_fastio_alloc that will act as a reference to the underlying memory resource, as well as store its own metadata state to describe the resource. This means that the underlying memory resource is shared and may be accessed directly by multiple operators, but the metadata that each operator uses to describe the resources is copied into local state and not otherwise shared between operators.

Publishing a Fast I/O allocation with nvidia_clara_fastio__publish copies the allocation’s metadata and underlying memory reference to the context using a unique string to identify the publication. After publication, the same string identifier can then be used by other operators executing in the pipeline to call nvidia_clara_fastio__get, which returns an nvidia_clara_fastio_alloc that will reference the same underlying memory resource and has its metadata initialized with a copy of the state that was published.

Metadata published to the context is immutable, and any further updates to metadata in local allocation objects will not be reflected in previous context publications of that allocation. Rather than support mutable metadata, allocations can instead be published to the context any number or times, with potentially different metadata state each time, so long as each publication uses a different unique identifier.

After calling nvidia_clara_fastio__publish, the nvidia_clara_fastio_alloc object that is published still remains valid in the calling process. It can still be used to access the memory resource and local metadata for that allocation, and it still needs to be released by calling nvidia_clara_fastio__release.

see also: Fast I/O Allocations

Parameters

  • context

    type: nvidia_clara_fastio_context *

    direction: In

    Pointer to the Fast I/O Context.

  • alloc

    type: nvidia_clara_fastio_alloc *

    direction: In

    Pointer to the Fast I/O allocation to publish.

  • name

    type: const char *

    direction: In

    Name to use for the publication.

Return

Returns zero if successful; otherwise a non-zero value.

Copy
Copied!
            

result nvidia_clara_fastio__publish( IN nvidia_clara_fastio_context *context, IN nvidia_clara_fastio_alloc *alloc, IN const char *name );

Allocations that were previously published to a Fast I/O context using nvidia_clara_fastio__publish can be retrieved by any other operator in the same pipeline job by calling nvidia_clara_fastio__get using the same identifier that was used to publish it. This will return a new nvidia_clara_fastio_alloc handle that references the same underlying memory resource and has its metadata initialized with a copy of the metadata that was published.

The allocations returned by this function act identically to those returned by nvidia_clara_fastio__create, and do not have any restrictions on their use. The metadata of these allocations can be modified, they can be published to the context, and it is the caller’s responsibility to call one of nvidia_clara_fastio__release or nvidia_clara_fastio__free in order to release the local references to the resource before returning from the execute callback.

see also: Fast I/O Allocations

Parameters

  • context

    type: nvidia_clara_fastio_context *

    direction: In

    Pointer to the Fast I/O Context.

  • name

    type: const char *

    direction: In

    Name of the publication to retrieve.

  • alloc_out

    type: nvidia_clara_fastio_alloc **

    direction: Out

    Pointer to the retrieved Fast I/O allocation.

Return

Returns zero if successful; otherwise a non-zero value.

Copy
Copied!
            

result nvidia_clara_fastio__get( IN nvidia_clara_fastio_context *context, IN const char *name, OUT nvidia_clara_fastio_alloc **alloc_out );

© Copyright 2018-2021, NVIDIA Corporation. All rights reserved. Last updated on Feb 1, 2023.