The Clara Pipeline Definition Language allows data scientists to define pipelines intended to be executed on the NVIDIA CLARA platform. Wikipedia defines declarative programming languages as “high-level program that describes what a computation should perform” or “a high-level program that describes what a computation should perform”.
- DAG based declaration of execution flow.
- Container native. Each DAG node is defined as a container.
- Based on YAML, with language support in all major IDE applications.
- Pipelines can be parameterized using run-time template resolution.
- Document linking, enables reuse of components across pipeline definitions.
9.1.1.1. Pipelines
Pipelines are created using the Clara Pipeline Definition Language and execute by Clara Deploy SDK. A pipeline consists of one or more operators, and it responsible for the coordination and execution flow of the operators. Pipelines are the top-level execution object, akin to processes.
See pipelines for more information.
9.1.1.2. Services
Services are created using the Clara Pipeline Definition Language and provide run-time execution support to pipeline operators. Services are started prior to a pipeline job starting. If a member operator of a pipeline requires a specific service to be available, Clara Deploy SDK will ensure that the service has been successfully deployed prior to starting the job. A service’s lifetime is at least as long as the lifetime of any pipeline utilizing the service. Unlike an operator, a service can be utilized by multiple pipeline job operators simultaneously and can exist beyond the lifetime of the pipeline which caused the service to be started.
See services for more information.
Notice: Pipeline services are deprecated as of Clara Deploy SDK v0.7.1 and support for pipeline services, including the ability to utilize them as part of a pipeline job, is expected to be dropped in a future release.
9.1.1.3. Operators
Operators are created using the Clara Pipeline Definition Language and execute as part of a pipeline’s execution. Operators read, write, and/or transform data as part of their execution. Operators can leverage services once they have declared a dependency on them. Operators are most like function or methods.
See operators for more information.