7.7. Clara Pipeline Fast I/O Entries
FastIO entries provide an interface that abstracts shared memory data management and access without having to explicitly manage them. FastIO input (or output) entries are generated when the first container in the pipeline tries to access an input (or output) in the pipeline definition of type array
, string
or any Clara Primitive (uint8
, uint16
, uint32
, uint64
, int8
, int16
, int32
, int64
, float16
, float32
, float64
).
FastIO output entries can be dynamic or well-defined.
A *well-defined output_ entry has positive integers specified in the
shape
when declared asarray
, or when declared as a Clara Primitive type. The following snippet shows two well-defined outputs from the first operator in the pipelineoperator-static
: .. code-block:: ymlapi-version: 0.5.0 orchestrator: Clara name: fastio-example-pipeline operators: - name: operator-static
description: Output two FastIO entries. container:
image: my-docker-image tag: my-tag
input: - path: /input
type: stream element-type: dicom
output: - name: dicom-data-in-shared-memory
type: array element-type: float32 shape: [3, 224, 255, 127]
name: my-second-fastio-output type: float32
the first output named
dicom-data-in-shared-memory
is a 4-dimensionalarray
ofshape = [3, 224, 255, 127]
where elements are of typefloat32
(4 byte floating point numbers);the second output named
my-second-fastio-output
is a single number of typefloat32
, which is effectively anarray
withshape = [1]
andelement-type = float32
.
A *dynamic output entry_ is either an
array
with-1
or0
entries in the shape, or an output of typestring
. The following snippet shows two dynamic outputs from the first operator in the pipelineoperator1
api-version: 0.5.0 orchestrator: Clara name: fastio-example-pipeline operators: - name: operator-dynamic description: Output two FastIO entries. container: image: my-docker-image tag: my-tag input: - path: /input type: stream element-type: dicom output: - name: dicom-data-in-shared-memory type: array element-type: float32 shape: [3, -1, -1, -1] - name: my-second-fastio-output type: string
the first output named
dicom-data-in-shared-memory
is a 4-dimensional array where the first dimension is well-defined (3
) whereas the rest of the dimensions can be changed duringoperator-dynamic
’s runtime.the second output is a
string
, which is effectively anarray
ofshape = [-1]
andelement-type = uint8
.
Both outputs are expected to be allocated during the runtime of the
operator-dynamic
container via the mutator function ``nvidia_clara_fastio_entry__update_shape` <update-shape>`. If all dimensions are well-defined, then one use the function ``nvidia_clara_fastio_entry__allocate` <allocate>` to create a shared memory allocation for the FastIO entry where the desired contents may be written and read (see FastIO Entry Automatic and Manual Allocation for more details).
FastIO input entries must match the type definition of FastIO output entries exactly, therefore can also be dynamic of well-defined. The snippet below shows two operators, where the output of one is the input to another.
api-version: 0.5.0
orchestrator: Clara
name: fastio-example-pipeline
operators:
- name: operator-dynamic-out
description: Output dynamic FastIO entry.
container:
image: my-docker-image-out
tag: my-tag
input:
- path: /input
type: stream
element-type: dicom
output:
- name: dicom-data-in-shared-memory
type: array
element-type: float32
shape: [3, -1, -1, -1]
- name: operator-dynamic-in
description: Output two FastIO entries.
container:
image: my-docker-image-in
tag: my-tag
input:
- from: operator-dynamic-out
name: dicom-data-in-shared-memory
type: array
element-type: float32
shape: [3, -1, -1, -1]
7.7.3.1. Well-defined FastIO Entries are Automatically Allocated
If output FastIO entries are well-defined they are allocated automatically by the Clara Pipeline driver. They do not need to be allocated by ``nvidia_clara_fastio_entry__allocate` <allocate>` and any calls to ``nvidia_clara_fastio_entry__update_shape` <update-shape>` will have no effect.
If a call is made to ``nvidia_clara_fastio_entry__allocate` <allocate>` the allocation will be freed and reallocated, effectively erasing any existing contents in the allocation.
Input FastIO entries, if derived from a well-defined FastIO output entry, will always have an allocation and the user need only ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` to read the contents and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` when finished or before the operator exits.
7.7.3.2. Dynamic FastIO Entries Should Be Manually Allocated
If output FastIO entries are dynamic, it is the responsibility of the code in the operator to
update the shape via ``nvidia_clara_fastio_entry__update_shape` <update-shape>` to a shape that is well-defined,
create the allocation via ``nvidia_clara_fastio_entry__allocate` <allocate>`,
use ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` to map the allocation to the operator process,
write to the allocation,
and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` when done or at least before the operator exits.
Very importantly if one fails to unmap an output FastIO entry before the operator finishes, no downstream operator can map the same FastIO entry-backed allocation as an input.
If one fails to create an allocation for a FastIO output entry in an operator, all downstream operators using said output entry as their input will receive that FastIO input entry without an allocation and will (effectively) not be able to use it.
7.7.3.3. Freeing Allocations
FastIO entry cannot be manually released; the release of allocations any FastIO entries in the pipeline is managed by the Clara Pipeline driver. Specifically, all allocations and associated metadata are released once the Pod Manager container exits.
Output FastIO entries are read/write, whereas input FastIO entries are read-only. This means that only the outputting operator can write data to the shared memory allocation of a FastIO entry, whereas all downstream operators using this FastIO entry as input will only have read access. Read-only access is imperative for when FastIO is used as input since the same FastIO output can be used as the input to multiple operators downstream; if read-write access would be given to FastIO inputs many operators could update the same shared-memory location in a sequence determined by the operating system which could result in unpredictable results, dirty-reads, or race conditions.
Before reading or modifying the contents of the shared memory tied to a FastIO entry, one must use``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` and ``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>`.
``nvidia_clara_fastio_entry__map_allocation` <map-allocation>` should be used before reading from or writing to the allocation of a FastIO entry.
``nvidia_clara_fastio_entry__unmap_allocation` <unmap-allocation>` should be used after reading from or writing to the allocation of a FastIO entry, or at least before the operator completes.
Very importantly if one fails to unmap an output FastIO entry before the operator finishes, no downstream operator can map the same FastIO entry-backed allocation as an input.
Gets the name of a nvidia_clara_fastio_entry
. The name of an input or output FastIO entry takes the form <operator-name>/<output-name>
both when used to get the output or its corresponding input. For example, in the definition:
api-version: 0.5.0
orchestrator: Clara
name: fastio-example-pipeline
operators:
- name: operator-dynamic-out
description: Output dynamic FastIO entry.
container:
image: my-docker-image-out
tag: my-tag
input:
- path: /input
type: stream
element-type: dicom
output:
- name: dicom-data-in-shared-memory
type: array
element-type: float32
shape: [3, -1, -1, -1]
- name: operator-dynamic-in
description: Output two FastIO entries.
container:
image: my-docker-image-in
tag: my-tag
input:
- from: operator-dynamic-out
name: dicom-data-in-shared-memory
type: array
element-type: float32
shape: [3, -1, -1, -1]
When
nvidia_clara_fastio_entry__name
is called inside the operatoroperator-dynamic-out
the name will appear asoperator-dynamic-out/dicom-data-in-shared-memory
.When
nvidia_clara_fastio_entry__name
is called inside the operatoroperator-dynamic-in
the name will appear asoperator-dynamic-out/dicom-data-in-shared-memory
which matches the output name exactly; this ensures the developer name consistency across operators.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
buffer
type:
char *
direction: In
Character buffer to which the name of a Fast IO entry will be written to.
buffer_size
type:
int
direction: In
The size of the incoming
buffer
parameter in bytes.len_out
type:
size_t *
direction: Out
The actual length of the name in bytes in the
buffer
.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__name(
IN nvidia_clara_fastio_entry *entry,
IN char *buffer,
IN int buffer_size,
OUT size_t *len_out)
Assigns a handle to the shape of a nvidia_clara_fastio_entry
to shape_out
as an array of integers, where length_out
indicated the number of elements in the array.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
shape_out
type:
int **
direction: Out
Handle to an array of integers with the shape of a FastIO entry.
length_out
type:
size_t *
direction: Out
The length/size of the
shape_out
in number of elements.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__shape(
IN nvidia_clara_fastio_entry *entry,
OUT int **shape_out,
OUT size_t *length_out)
Assigns a handle to element_type_out
to indicate the element-type of a nvidia_clara_fastio_entry
.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
element_type_out
type:
nvidia_clara_element_type *
direction: Out
The element-type of the items/contents of a FastIO entry.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__element_type(
IN nvidia_clara_fastio_entry *entry,
OUT nvidia_clara_element_type *element_type_out)
Assigns a handle to size_bytes_out
containing the number of bytes as estimated from the shape and element-type of the nvidia_clara_fastio_entry
.
For well-defined FastIO entries the size of bytes esimated using nvidia_clara_fastio_entry__size_bytes
should be the same as the number of bytes returned by ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>`.
However, dynamic FastIO entries may have a well-defined shape, after updates to its shape using ``nvidia_clara_fastio_entry__update_shape` <update-shape>`, but no allocation to back the entry up since perhaps ``nvidia_clara_fastio_entry__allocate` <allocate>` might not have been called yet.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
size_bytes_out
type:
size_t *
direction: Out
The size in bytes of a FastIO entry as described by its element-type and shape.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__size_bytes(
IN nvidia_clara_fastio_entry *entry,
OUT size_t *size_bytes_out)
Assigns a handle to dynamic_indices_out
that holds the indices of the shape nvidia_clara_fastio_entry
that are dynamic (changeable via ``nvidia_clara_fastio_entry__update_shape` <update-shape>`). dynamic_indices_length_out
indicated the number of elements in dynamic_indices_out
.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
dynamic_indices_out
type:
int **
direction: Out
A handle to an array of integers indicating the dimensions in th shape that are dynamic (resizable at runtime).
dynamic_indices_length_out
type:
int *
direction: Out
The length in number of items of
dynamic_indices_out
.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__dynamic_indices(
IN nvidia_clara_fastio_entry *entry,
OUT int **dynamic_indices_out,
OUT int *dynamic_indices_length_out)
Returns a handle is_dynamic_out
to a boolean value (represented as an integer 0 = false
, 1 = true
) to indicate whether a nvidia_clara_fastio_entry
is dynamic. is_dynamic_out
is true
if any of the values of shape
in the pipeline definition are -1
or 0
, or the type
is string
.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
is_dynamic_out
type:
int *
direction: Out
A boolean value represented as an integer (
0 => false
,1 => true
) to indicate whether the FastIO entry has dynamic dimensions that are resizable at runtime.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__is_dynamic(
IN nvidia_clara_fastio_entry *entry,
OUT int* is_dynamic_out)
Updates the shape of a nvidia_clara_fastio_entry
, where dimensions_to_update
holds the indices in the shape to be updated, and values_to_update
holds the corresponding values to be updated. The argument length
indicates the length of both dimensions_to_update
and values_to_update
, and the number of items in each must match length
.
Important: If a list of non-dynamic indices is passed to dimensions_to_update
they will not be updated. For example, if shape = [3, -1, 224, -1]
and the user passes
dimensions_to_update = [0, 1, 3]
values_to_update = [4, 224, 224]
length = 3
the resulting shape will be shape = [3, 224, 224, 224]
. Note that the shape element at index 0
was not updated, since it was well-defined at pipeline definition.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
dimensions_to_update
type:
int *
direction: In
An array containing the indices of the shape (enumerated from zero) that will be updated.
values_to_update
type:
int *
direction: In
An array containing the values of the shape (enumerated from zero) that will be updated that correspond to the indices (
dimensions_to_update
) passed in.length
type:
int *
direction: In
The length in elements of the
dimensions_to_update
andvalues_to_update
(these must be the same and metchlength
).
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__update_shape(
IN nvidia_clara_fastio_entry *entry,
IN int *dimensions_to_update,
IN int *values_to_update,
IN int *length)
Returns the type of IOAccess
the operator has on the allocation of a nvidia_clara_fastio_entry
.
I/O access is read-only for FastIO input entries, and
read/write for for FastIO output entries.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
access_out
type:
int *
direction: Out
The type of access one has to the FastIO entry. This can be
1
for “read-only”2
for “read/write”
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__access(
IN nvidia_clara_fastio_entry *entry,
OUT int *access_out)
Creates a shared memory allocation for a nvidia_clara_fastio_entry
. The allocation will only be created if all dimensions of the shape are well-defined, that is, positive integers; if not, the function will return a non-zero value. The size of the allocation will be assigned based element-type size and the shape.
Important: Multiple calls to nvidia_clara_fastio_entry__allocate
will result in reallocation, and therefore the erasing of all existing data in the allocation.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__allocate(
IN nvidia_clara_fastio_entry *entry)
Assigns a boolean value (represented as an integer 0 = false
, 1 = true
) to the handle is_allocated_out
to indicate whether a nvidia_clara_fastio_entry
has a shared memory allocation.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
is_allocated_out
type:
int *
direction: In
Handle to the Fast I/O entry.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__is_allocated(
IN nvidia_clara_fastio_entry *entry,
OUT int *is_allocated_out)
Maps an allocation to the current operator process, and assigns a handle to the allocation to allocation_out
and the allocation size in bytes to alloc_byte_size_out
.
Important: alloc_byte_size_out
is not guaranteed to match the size estimated by ``nvidia_clara_fastio_entry__size_bytes` <size-in-bytes>`; for example if one uses the following sequence of calls
nvidia_clara_fastio_entry__update_shape(...); // will update the shape, but no allocation exists
nvidia_clara_fastio_entry__allocate(...); // the size of the allocation matches `nvidia_clara_fastio_entry__size_bytes`
nvidia_clara_fastio_entry__update_shape(...); // the size of the allocation no longer matches `nvidia_clara_fastio_entry__size_bytes`
nvidia_clara_fastio_entry__allocate(...); // the size of the allocation again matches `nvidia_clara_fastio_entry__size_bytes`
The comments in the code above shows when the estimated size of the nvidia_clara_fastio_entry
matches the size of its allocation.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
allocation_out
type:
void **
direction: In
Handle to the shared memory allocation of a FastIO entry mapped into the current process.
alloc_byte_size_out
type:
int *
direction: In
The size in bytes of the shared memory allocation of a FastIO entry.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__map_allocation(
IN nvidia_clara_fastio_entry *entry,
OUT void **allocation_out,
OUT int *alloc_byte_size_out)
Unmaps the allocation of a nvidia_clara_fastio_entry
from an operator process. This operation (nvidia_clara_fastio_entry__unmap_allocation
) should be performed only if an allocation has been previously mapped via ``nvidia_clara_fastio_entry__map_allocation` <map-allocation>`, otherwise an non-zero code is returned.
Important: Mapped allocations must be unmapped if access is read/write on that allocation (this applies to FastIO output entries); if one fails to unmap a read/write access allocation (FastIO output entry) before the operator process exits, downstream operators will fail to acquire a read-only lock on the allocation (to use as a FastIO input entry), rendering the allocation effectively unusable.
Parameters
entry
type:
nvidia_clara_fastio_entry *
direction: In
Handle to the Fast I/O entry.
allocation
type:
void *
direction: In
Handle to the shared memory allocation of a FastIO entry to be unmapped from the current process.
Return
Returns zero if successful; otherwise a non-zero value.
result nvidia_clara_fastio_entry__unmap_allocation(
IN nvidia_clara_fastio_entry *entry,
IN void *allocation)