8.12. Clara Pipeline Fast I/O Context

While the clara.Payload traditionally provides disk-based input and output support to applications, it is often the case that medical imaging pipelines will be operating on very large data sets, sharing and/or passing data between the operators in the pipeline, and swapping this data to and from disk between each operator is inefficient in both runtime and resource usage.

To avoid the inefficient use of disk-based data passing, the clara.Payload also provides a Fast I/O Context, clara.FastIOContext, which is an object that provides access to memory resources that are accessible by all operators running within the same pipeline, and which can be used for for efficient, zero-copy sharing and passing of data between operators.

The Fast I/O Context provides not just the means to allocate Fast I/O memory resources (FastIOContext.create), but also provides a mechanism to pass these resources between multiple operators in a pipeline using basic string identifiers (e.g. using FastIOAllocation.publish and FastIOContext.get)

A clara.FastIOContext object can be retrieved from the Payload.fastio_context property within the clara.Payload object that was provided to the application as part of the Clara Pipeline Operator Life-Cycle execution event callback. This object is only valid throughout the lifetime of the application supplied execution callback handler.

see also: Fast I/O Allocations, Payload, and Driver Callbacks

Allocates a Fast I/O memory resource of the requested size. Returns a clara.FastIOAllocation object to identify the allocation.

Fast I/O allocations returned by this function are initially only accessible by the calling process. In order to make a Fast I/O allocation available to other operators in the pipeline, the allocating process must publish the allocation to the Fast I/O context using FastIOAllocation.publish, after which other operators may access and import a local reference to that same resource using FastIOContext.get.

see also: Fast I/O Allocations

Parameters

• size

  type: int

  direction: In

  Size, in bytes, of the requested allocation.

Return

Returns a clara.FastIOAllocation object for the new allocation.

            

            
def create(self, size)
        

Allocations that were previously published to a Fast I/O context using FastIOAllocation.publish can be retrieved by any other operator in the same pipeline job by calling FastIOContext.get using the same identifier that was used to publish it. This will return a new clara.FastIOAllocation object that references the same underlying memory resource and has its metadata initialized with a copy of the metadata that was published.

The allocations returned by this function act identically to those returned by FastIOContext.create, and do not have any restrictions on their use. The metadata of these objects can be modified, the allocations can be published to the context, and they can be used to free the underlying memory resources with FastIOAllocation.free.

see also: Fast I/O Allocations

Parameters

• name

  type: string

  direction: In

  Name of the publication to retrieve.

Return

Returns a clara.FastIOAllocation object for the retrieved allocation.

            

            
def get(self, name)
        

This utility function publishes Numpy arrays directly to a Fast I/O context without needing to manually create, initialize, and publish a clara.FastIOAllocation object.

This function does the following:

1. Allocates a Fast I/O memory region large enough for the provided numpy array.

2. Sets the metadata (type/dimensions) of the Fast I/O allocation to match that of the array.

3. Copies the contents of the array to the Fast I/O memory.

4. Publishes the allocation to the context using the given idendifier.

Parameters

• array

  type: numpy.array

  direction: In

  Numpy array to publish.

• name

  type: string

  direction: In

  Name to use for the array’s Fast I/O publication.

Return

No return value.

            

            
def publish_array(self, array, name)