9.20. Syntax & Grammar¶
The Clara Pipeline Definition Language (or CPDL) is a specification for developing pipeline configurations which can be executed by Clara Deploy SDK as pipeline jobs. CPDL is a domain specific implementation of YAML; with YAML providing the syntax rules and CPDL the grammar rules.
Clara Deploy SDK pipeline definitions are broken down into three parts: operators, pipelines, and services. Each part is responsible for a specific functionality needed to enable developers to provide a wide range of execution pipelines.
9.20.1. YAML Basics¶
YAML is a name-value pair configuration language, designed to be both human and machine readable. Each pair is often referred to as a “property”. All name-value pairs will have one of the following formats:
220.127.116.11. Scalar Value:¶
18.104.22.168. List Value¶
property-name: - property-value-1 - property-value-2
22.214.171.124. Object Value¶
property-name: sub-property-1-name: property-value sub-property-2-name: property-value
126.96.36.199. Array Value¶
property-name: ['array', 'property', 'value']
188.8.131.52. Scalar Value with New Line Character¶
Notice that each value entry ends with a new line character. When the value itself contains a new line character, double-quotes
" can be used to bookend the value to allow for new line characters. The value will still require a new line character to be present after the closing double-quote character. Only scalar values can be double-quoted like this.
property-name: "This is a scalar value with a new line character contained inside of it."
184.108.40.206. Indentation and White Space Matters¶
Consecutive lines with the same indentation are considered to be properties of the first less indented line preceeding them.
When a document mixes spaces and tabs, the results are undefined and are dependant on the YAML interpreter.
It is recommended that indentation using spaces, and indentation using tabs not be mixed in the same document. Importing a tab indented document into a space indented document, or vice-versa, using the CPDL import declaration is safe and supported.
See import for details.
Documents have one or more properties. Each property can be either a list, object, or scalar value. The comments below attempt to show examples of each type in use as part of a YAML document.
## The name of the property is on the left-hand side of the colon `:` ## The value of the property is on the right-hand side of the colon `:`. name: example-pipeline description: Example pipeline definition to demonstrate how YAML can be used. ## The "operators" property is a list of values. operators: ## Each value in the list is an object value. - name: operator-1 ## "container" is a property of the object value contained in the "operators" list. container: image: clara/example/operator-1 tag: v1.0.0 input: - path: /in output: - path: /out - name: operator-2 container: image: clara/example/operator-2 tag: v1.0.0 input: - from: operator-1 path: /in output: - path: /out
See document for details.
9.20.2. Operator Definitions¶
Operator definitions provide a way to describe a containerized application. Operator definitions are intended to be abstract contracts which enable reuse of containerized applications across multiple pipelines.
Operators are akin to functions in traditional software development languages. Like functions, operators provide a signature or set of input and output values for other operators to consume. Like functions, they provide data processing (the containerized application itself) so long as the requirements of the signature are met.
See operator for details.
See container for details.