6.12. Syntax & Grammar

The Clara Pipeline Definition Language (or CPDL) is a specification for developing pipeline configurations which can be executed by Clara Deploy SDK as pipeline jobs. CPDL is a domain specific implementation of YAML; with YAML providing the syntax rules and CPDL the grammar rules.

Clara Deploy SDK pipeline definitions are broken down into three parts: operators, pipelines, and services. Each part is responsible for a specific functionality needed to enable developers to provide a wide range of execution pipelines.

6.12.1. YAML Basics

YAML is a name-value pair configuration language, designed to be both human and machine readable. Each pair is often referred to as a “property”. All name-value pairs will have one of the following formats:

6.12.1.1. Scalar Value:

property-name: property-value

6.12.1.2. List Value

property-name:
   - property-value-1
   - property-value-2

6.12.1.3. Object Value

property-name:
  sub-property-1-name: property-value
  sub-property-2-name: property-value

6.12.1.4. Array Value

property-name: ['array', 'property', 'value']

6.12.1.5. Scalar Value with New Line Character

Notice that each value entry ends with a new line character. When the value itself contains a new line character, double-quotes " can be used to bookend the value to allow for new line characters. The value will still require a new line character to be present after the closing double-quote character. Only scalar values can be double-quoted like this.

property-name: "This is a scalar value
      with a new line character
      contained inside of it."

6.12.1.6. Comments

YAML comments begin with a hash # character and end with a new line character.

## This is a comment, not a value, and ignored by the interpreter
example: This is not a comment, and it is read by the interpereter.

6.12.1.7. Indentation and White Space Matters

Consecutive lines with the same indentation are considered to be properties of the first less indented line preceeding them.

When a document mixes spaces and tabs, the results are undefined and are dependant on the YAML interpreter.

It is recommended that indentation using spaces, and indentation using tabs not be mixed in the same document. Importing a tab indented document into a space indented document, or vice-versa, using the CPDL import declaration is safe and supported.

See import for details.

6.12.1.7.1. Example

Documents have one or more properties. Each property can be either a list, object, or scalar value. The comments below attempt to show examples of each type in use as part of a YAML document.

## The name of the property is on the left-hand side of the colon `:`
## The value of the property is on the right-hand side of the colon `:`.
name: example-pipeline
description: Example pipeline definition to demonstrate how YAML can be used.
## The "operators" property is a list of values.
operators:
  ## Each value in the list is an object value.
  - name: operator-1
    ## "container" is a property of the object value contained in the "operators" list.
    container:
      image: clara/example/operator-1
      tag: v1.0.0
    input:
      - path: /in
    output:
      - path: /out
  - name: operator-2
    container:
      image: clara/example/operator-2
      tag: v1.0.0
    input:
      - from: operator-1
        path: /in
    output:
      - path: /out

See document for details.

6.12.2. Operator Definitions

Operator definitions provide a way to describe a containerized application. Operator definitions are intended to be abstract contracts which enable reuse of containerized applications across multiple pipelines.

Operators are akin to functions in traditional software development languages. Like functions, operators provide a signature or set of input and output values for other operators to consume. Like functions, they provide data processing (the containerized application itself) so long as the requirements of the signature are met.

See operator for details.

See container for detalis.

6.12.3. Service Definitions

Service definitions provide a way to describe a containerized application. Service definitions are intended to be abstract contracts which enable reuse of containerized services across multiple pipeline operators.

See service for details.

See container for detalis.