7.7. Clara Pipeline Fast I/O Entries

FastIO entries provide an interface that abstracts shared memory data management and access without having to explicitly manage them. FastIO input (or output) entries are generated when the first container in the pipeline tries to access an input (or output) in the pipeline definition of type array, string or any Clara Primitive (uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32, float64).

7.7.1. Output Entries

FastIO output entries can be dynamic or well-defined.

  • A *well-defined output_ entry has positive integers specified in the shape when declared as array, or when declared as a Clara Primitive type. The following snippet shows two well-defined outputs from the first operator in the pipeline operator-static: .. code-block:: yml

    api-version: 0.5.0 orchestrator: Clara name: fastio-example-pipeline operators: - name: operator-static

    description: Output two FastIO entries. container:

    image: my-docker-image tag: my-tag

    input: - path: /input

    type: stream element-type: dicom

    output: - name: dicom-data-in-shared-memory

    type: array element-type: float32 shape: [3, 224, 255, 127]

    • name: my-second-fastio-output type: float32
    • the first output named dicom-data-in-shared-memory is a 4-dimensional array of shape = [3, 224, 255, 127] where elements are of type float32 (4 byte floating point numbers);
    • the second output named my-second-fastio-output is a single number of type float32, which is effectively an array with shape = [1] and element-type = float32.
  • A *dynamic output entry_ is either an array with -1 or 0 entries in the shape, or an output of type string. The following snippet shows two dynamic outputs from the first operator in the pipeline operator1

    api-version: 0.5.0
    orchestrator: Clara
    name: fastio-example-pipeline
    operators:
    - name: operator-dynamic
      description: Output two FastIO entries.
      container:
        image: my-docker-image
        tag: my-tag
      input:
      - path: /input
        type: stream
        element-type: dicom
      output:
      - name: dicom-data-in-shared-memory
        type: array
        element-type: float32
        shape: [3, -1, -1, -1]
      - name: my-second-fastio-output
        type: string
    
    • the first output named dicom-data-in-shared-memory is a 4-dimensional array where the first dimension is well-defined (3) whereas the rest of the dimensions can be changed during operator-dynamic’s runtime.
    • the second output is a string, which is effectively an array of shape = [-1] and element-type = uint8.

    Both outputs are expected to be allocated during the runtime of the operator-dynamic container via the mutator function ``nvidia_clara_fastio_entry__update_shape` <update-shape>`. If all dimensions are well-defined, then one use the function ``nvidia_clara_fastio_entry__allocate` <allocate>` to create a shared memory allocation for the FastIO entry where the desired contents may be written and read (see FastIO Entry Automatic and Manual Allocation for more details).

7.7.2. Input Entries

FastIO input entries must match the type definition of FastIO output entries exactly, therefore can also be dynamic of well-defined. The snippet below shows two operators, where the output of one is the input to another.

api-version: 0.5.0
  orchestrator: Clara
  name: fastio-example-pipeline
  operators:
  - name: operator-dynamic-out
    description: Output dynamic FastIO entry.
    container:
      image: my-docker-image-out
      tag: my-tag
    input:
    - path: /input
      type: stream
      element-type: dicom
    output:
    - name: dicom-data-in-shared-memory
      type: array
      element-type: float32
      shape: [3, -1, -1, -1]
  - name: operator-dynamic-in
    description: Output two FastIO entries.
    container:
      image: my-docker-image-in
      tag: my-tag
    input:
    - from: operator-dynamic-out
      name: dicom-data-in-shared-memory
      type: array
      element-type: float32
      shape: [3, -1, -1, -1]

7.7.3. FastIO Entry Automatic and Manual Allocation

7.7.3.1. Well-defined FastIO Entries are Automatically Allocated

If output FastIO entries are well-defined they are allocated automatically by the Clara Pipeline driver. They do not need to be allocated by ``nvidia_clara_fastio_entry__allocate` <allocate>` and any calls to ``nvidia_clara_fastio_entry__update_shape` <update-shape>` will have no effect.

  • If a call is made to ``nvidia_clara_fastio_entry__allocate` <allocate>` the allocation will be freed and reallocated, effectively erasing any existing contents in the allocation.

Input FastIO entries, if derived from a well-defined FastIO output entry, will always have an allocation and the user need only ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` to read the contents and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` when finished or before the operator exits.

7.7.3.2. Dynamic FastIO Entries Should Be Manually Allocated

If output FastIO entries are dynamic, it is the responsibility of the code in the operator to

  1. update the shape via ``nvidia_clara_fastio_entry__update_shape` <update-shape>` to a shape that is well-defined,
  2. create the allocation via ``nvidia_clara_fastio_entry__allocate` <allocate>`,
  3. use ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` to map the allocation to the operator process,
  4. write to the allocation,
  5. and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` when done or at least before the operator exits.

Very importantly if one fails to unmap an output FastIO entry before the operator finishes, no downstream operator can map the same FastIO entry-backed allocation as an input.

If one fails to create an allocation for a FastIO output entry in an operator, all downstream operators using said output entry as their input will receive that FastIO input entry without an allocation and will (effectively) not be able to use it.

7.7.3.3. Freeing Allocations

FastIO entry cannot be manually released; the release of allocations any FastIO entries in the pipeline is managed by the Clara Pipeline driver. Specifically, all allocations and associated metadata are released once the Pod Manager container exits.

7.7.4. FastIO Entry I/O Access

Output FastIO entries are read/write, whereas input FastIO entries are read-only. This means that only the outputting operator can write data to the shared memory allocation of a FastIO entry, whereas all downstream operators using this FastIO entry as input will only have read access. Read-only access is imperative for when FastIO is used as input since the same FastIO output can be used as the input to multiple operators downstream; if read-write access would be given to FastIO inputs many operators could update the same shared-memory location in a sequence determined by the operating system which could result in unpredictable results, dirty-reads, or race conditions.

Before reading or modifying the contents of the shared memory tied to a FastIO entry, one must use``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>`.

  • ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` should be used before reading from or writing to the allocation of a FastIO entry.
  • ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` should be used after reading from or writing to the allocation of a FastIO entry, or at least before the operator completes.

Very importantly if one fails to unmap an output FastIO entry before the operator finishes, no downstream operator can map the same FastIO entry-backed allocation as an input.

7.7.5. Name

Gets the name of a nvidia_clara_fastio_entry. The name of an input or output FastIO entry takes the form <operator-name>/<output-name> both when used to get the output or its corresponding input. For example, in the definition:

api-version: 0.5.0
  orchestrator: Clara
  name: fastio-example-pipeline
  operators:
  - name: operator-dynamic-out
    description: Output dynamic FastIO entry.
    container:
      image: my-docker-image-out
      tag: my-tag
    input:
    - path: /input
      type: stream
      element-type: dicom
    output:
    - name: dicom-data-in-shared-memory
      type: array
      element-type: float32
      shape: [3, -1, -1, -1]
  - name: operator-dynamic-in
    description: Output two FastIO entries.
    container:
      image: my-docker-image-in
      tag: my-tag
    input:
    - from: operator-dynamic-out
      name: dicom-data-in-shared-memory
      type: array
      element-type: float32
      shape: [3, -1, -1, -1]
  • When nvidia_clara_fastio_entry__name is called inside the operator operator-dynamic-out the name will appear as operator-dynamic-out/dicom-data-in-shared-memory.
  • When nvidia_clara_fastio_entry__name is called inside the operator operator-dynamic-in the name will appear as operator-dynamic-out/dicom-data-in-shared-memory which matches the output name exactly; this ensures the developer name consistency across operators.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • buffer

    type: char *

    direction: In

    Character buffer to which the name of a Fast IO entry will be written to.

  • buffer_size

    type: int

    direction: In

    The size of the incoming buffer parameter in bytes.

  • len_out

    type: size_t *

    direction: Out

    The actual length of the name in bytes in the buffer.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__name(
    IN nvidia_clara_fastio_entry *entry,
    IN char *buffer,
    IN int buffer_size,
   OUT size_t *len_out)

7.7.6. Shape

Assigns a handle to the shape of a nvidia_clara_fastio_entry to shape_out as an array of integers, where length_out indicated the number of elements in the array.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • shape_out

    type: int **

    direction: Out

    Handle to an array of integers with the shape of a FastIO entry.

  • length_out

    type: size_t *

    direction: Out

    The length/size of the shape_out in number of elements.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__shape(
    IN nvidia_clara_fastio_entry *entry,
   OUT int **shape_out,
   OUT size_t *length_out)

7.7.7. Element Type

Assigns a handle to element_type_out to indicate the element-type of a nvidia_clara_fastio_entry.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • element_type_out

    type: nvidia_clara_element_type *

    direction: Out

    The element-type of the items/contents of a FastIO entry.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__element_type(
    IN nvidia_clara_fastio_entry *entry,
   OUT nvidia_clara_element_type *element_type_out)

7.7.8. Size in Bytes

Assigns a handle to size_bytes_out containing the number of bytes as estimated from the shape and element-type of the nvidia_clara_fastio_entry.

For well-defined FastIO entries the size of bytes esimated using nvidia_clara_fastio_entry__size_bytes should be the same as the number of bytes returned by ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>`.

However, dynamic FastIO entries may have a well-defined shape, after updates to its shape using ``nvidia_clara_fastio_entry__update_shape` <update-shape>`, but no allocation to back the entry up since perhaps ``nvidia_clara_fastio_entry__allocate` <allocate>` might not have been called yet.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • size_bytes_out

    type: size_t *

    direction: Out

    The size in bytes of a FastIO entry as described by its element-type and shape.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__size_bytes(
    IN nvidia_clara_fastio_entry *entry,
   OUT size_t *size_bytes_out)

7.7.9. Dynamic Indices

Assigns a handle to dynamic_indices_out that holds the indices of the shape nvidia_clara_fastio_entry that are dynamic (changeable via ``nvidia_clara_fastio_entry__update_shape` <update-shape>`). dynamic_indices_length_out indicated the number of elements in dynamic_indices_out.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • dynamic_indices_out

    type: int **

    direction: Out

    A handle to an array of integers indicating the dimensions in th shape that are dynamic (resizable at runtime).

  • dynamic_indices_length_out

    type: int *

    direction: Out

    The length in number of items of dynamic_indices_out.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__dynamic_indices(
    IN nvidia_clara_fastio_entry *entry,
   OUT int **dynamic_indices_out,
   OUT int *dynamic_indices_length_out)

7.7.10. Is Dynamic

Returns a handle is_dynamic_out to a boolean value (represented as an integer 0 = false, 1 = true) to indicate whether a nvidia_clara_fastio_entry is dynamic. is_dynamic_out is true if any of the values of shape in the pipeline definition are -1 or 0, or the type is string.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • is_dynamic_out

    type: int *

    direction: Out

    A boolean value represented as an integer (0 => false, 1 => true) to indicate whether the FastIO entry has dynamic dimensions that are resizable at runtime.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__is_dynamic(
    IN nvidia_clara_fastio_entry *entry,
   OUT int* is_dynamic_out)

7.7.11. Update Shape

Updates the shape of a nvidia_clara_fastio_entry, where dimensions_to_update holds the indices in the shape to be updated, and values_to_update holds the corresponding values to be updated. The argument length indicates the length of both dimensions_to_update and values_to_update, and the number of items in each must match length.

Important: If a list of non-dynamic indices is passed to dimensions_to_update they will not be updated. For example, if shape = [3, -1, 224, -1] and the user passes

dimensions_to_update = [0, 1, 3]
values_to_update = [4, 224, 224]
length = 3

the resulting shape will be shape = [3, 224, 224, 224]. Note that the shape element at index 0 was not updated, since it was well-defined at pipeline definition.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • dimensions_to_update

    type: int *

    direction: In

    An array containing the indices of the shape (enumerated from zero) that will be updated.

  • values_to_update

    type: int *

    direction: In

    An array containing the values of the shape (enumerated from zero) that will be updated that correspond to the indices (dimensions_to_update) passed in.

  • length

    type: int *

    direction: In

    The length in elements of the dimensions_to_update and values_to_update (these must be the same and metch length).

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__update_shape(
    IN nvidia_clara_fastio_entry *entry,
    IN int *dimensions_to_update,
    IN int *values_to_update,
    IN int *length)

7.7.12. Access

Returns the type of IOAccess the operator has on the allocation of a nvidia_clara_fastio_entry.

  • I/O access is read-only for FastIO input entries, and
  • read/write for for FastIO output entries.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • access_out

    type: int *

    direction: Out

    The type of access one has to the FastIO entry. This can be

    • 1 for “read-only”
    • 2 for “read/write”

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__access(
    IN nvidia_clara_fastio_entry *entry,
    OUT int *access_out)

7.7.13. Allocate

Creates a shared memory allocation for a nvidia_clara_fastio_entry. The allocation will only be created if all dimensions of the shape are well-defined, that is, positive integers; if not, the function will return a non-zero value. The size of the allocation will be assigned based element-type size and the shape.

Important: Multiple calls to nvidia_clara_fastio_entry__allocate will result in reallocation, and therefore the erasing of all existing data in the allocation.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__allocate(
    IN nvidia_clara_fastio_entry *entry)

7.7.14. Is Allocated

Assigns a boolean value (represented as an integer 0 = false, 1 = true) to the handle is_allocated_out to indicate whether a nvidia_clara_fastio_entry has a shared memory allocation.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • is_allocated_out

    type: int *

    direction: In

    Handle to the Fast I/O entry.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__is_allocated(
    IN nvidia_clara_fastio_entry *entry,
   OUT int *is_allocated_out)

7.7.15. Map Allocation

Maps an allocation to the current operator process, and assigns a handle to the allocation to allocation_out and the allocation size in bytes to alloc_byte_size_out.

Important: alloc_byte_size_out is not guaranteed to match the size estimated by ``nvidia_clara_fastio_entry__size_bytes` <size-in-bytes>`; for example if one uses the following sequence of calls

nvidia_clara_fastio_entry__update_shape(...);   // will update the shape, but no allocation exists
nvidia_clara_fastio_entry__allocate(...); // the size of the allocation matches `nvidia_clara_fastio_entry__size_bytes`
nvidia_clara_fastio_entry__update_shape(...);   // the size of the allocation no longer matches `nvidia_clara_fastio_entry__size_bytes`
nvidia_clara_fastio_entry__allocate(...); // the size of the allocation again matches `nvidia_clara_fastio_entry__size_bytes`

The comments in the code above shows when the estimated size of the nvidia_clara_fastio_entry matches the size of its allocation.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • allocation_out

    type: void **

    direction: In

    Handle to the shared memory allocation of a FastIO entry mapped into the current process.

  • alloc_byte_size_out

    type: int *

    direction: In

    The size in bytes of the shared memory allocation of a FastIO entry.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__map_allocation(
    IN nvidia_clara_fastio_entry *entry,
   OUT void **allocation_out,
   OUT int *alloc_byte_size_out)

7.7.16. Unmap Allocation

Unmaps the allocation of a nvidia_clara_fastio_entry from an operator process. This operation (nvidia_clara_fastio_entry__unmap_allocation) should be performed only if an allocation has been previously mapped via ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>`, otherwise an non-zero code is returned.

Important: Mapped allocations must be unmapped if access is read/write on that allocation (this applies to FastIO output entries); if one fails to unmap a read/write access allocation (FastIO output entry) before the operator process exits, downstream operators will fail to acquire a read-only lock on the allocation (to use as a FastIO input entry), rendering the allocation effectively unusable.

Parameters

  • entry

    type: nvidia_clara_fastio_entry *

    direction: In

    Handle to the Fast I/O entry.

  • allocation

    type: void *

    direction: In

    Handle to the shared memory allocation of a FastIO entry to be unmapped from the current process.

Return

Returns zero if successful; otherwise a non-zero value.

result nvidia_clara_fastio_entry__unmap_allocation(
    IN nvidia_clara_fastio_entry *entry,
    IN void *allocation)