7.1. Clara Pipeline Driver Overview

The NVIDIA Clara Pipeline Driver is a wrapper library that

  • ensures that input and output dependencies between operators are respected,
  • and allows the user code to retrieve data from pipeline-defined input/output paths or shared memory variables, allocations, and contexts. It is provided as a library to be included as part of your pipeline stage’s worker process. Client libraries are available for Python and for C/C++ applications.

7.1.1. Fast I/O: Shared Memory Data Exchange Between Operators

Fast I/O comes in two different flavors:

  • Using Fast I/O Contexts and Fast I/O Allocations, where the user must manually manage shared memory contexts and allocations through the Fast I/O Context and Fast I/O Allocation APIs respectively.
  • Using Fast I/O Variables, available as of pipeline api-version 0.5.0 and up, where shared memory items are declared in the operator’s definition.

7.1.1.1. Fast I/O Contexts and Allocations

(Suitable only for pipeline ``api-version >= 0.4.0``)

Developers can employ Fast I/O Contexts and Allocations in pipeline api-version 0.4.0. In this paradigm, the user is provided with a Fast I/O Context generated by an implicit operator in Clara orchestrated pods, the Pod Manager. The Fast I/O context is mage available to all operators in the pipeline, and therefore any Fast I/O allocation published to the context is available for reading and writing to all operators.

7.1.1.1.1. Drawbacks and Risks

  1. The management of Fast I/O allocations (whether using C or Python API) is C-style, where allocations have to be created, allocated, published to a context, mapped (before updates can be made) and unmapped (after completing value updates) from an operator process. Failures to properly manage allocations will result in errors or memory leaks.
  2. Two operators that run concurrently while updating the same Fast I/O allocation will cause
    • one operator failing to map the allocation if both are trying to map the allocation for writing,
    • dirty reads if one operator maps the allocation for reading and the other for writing.
  3. Fast I/O allocations and Fast I/O contexts do not create a dependency graph between operators, therefore one must create a dummy disk-based input/output pair between operators, otherwise the orchestrator will schedule two dependent operators to run concurrently.

It is advisable that, unless there are very specific reasons to manually manage Fast I/O contexts and allocations, operators should employ the Fast I/O Variable API (next section).

7.1.1.2. Fast I/O Variables

(Suitable only for typed pipelines - ``api-version`` ``0.5.0`` and higher)

The Fast I/O variable API addresses the shortcomings of manually managed shared memory contexts and allocations (via the Fast I/O context and allocations API).

The Fast I/O variable API manages

  • the life-cycle of any shared memory entries,
  • their scope in the pipeline (allocation and deallocation),
  • the read/write access of the shared memory items (inputs always set to read-only and outputs to read/write),
  • and the dependencies between operators (i.e. an operator with a shared memory input will wait for the corresponding shared memory output of an upstream operator).

7.1.2. Python Client

7.1.3. Native Client

The Clara Pipeline Driver libraries are developed and targeted towards Ubuntu 18.04. Using other Linux variants is possible but not supported. This is true for the Python library as well as it makes extensive use of the native library.