Clara Holoscan Deploy 0.7.4
0.7.4

Clara Workflow Creation

This document describes how a workflow is defined in Clara Deploy SDK. A workflow can be created within a helm chart as a sequence of steps (stages). A step-by-step guide is provided to create a new workflow and deploy it within the Clara Deploy SDK. This document includes an example workflow that uses the “dicom-reader” provided by the Clara Deploy SDK, a user created container that performs the AI, and the dicom-writer container provided by the Clara Deploy SDK. Additionally, this document provides guidance on how to create a helm chart for any user container.

The clara-wf command line tool can be used to define a workflow within Clara, to update or create a Helm chart for a new or existing container, and for testing workflows locally.

Workflows have the following characteristics:

  • Workflow ID: A Workflow ID is created manually and is configured with a dicom-server configuration file.

  • name: User-defined workflow name. This name is used in dicom-server configuration.

  • stages: Containers that are used in the workflow. The sequence in which containers are used in the workflow is defined under “stages”.

  • waitlocks: Dependency of each stage on other stages. Lock files are automatically generated based on stage name.

  • ioFolders: Input of each stage in the workflow. Last entry is the output of the last stage of the workflow. The result of the last entry is sent to DICOM adapter.

  • args: Command line arguments that are supported for this workflow and defined in the helm chart for the container. An argument and its value are expressed within double quotes and picked up for execution at run time.

To Create a Helm Chart for User-Defined Containers

  1. Ensure the container is updated with clara driver callbacks as described in workflow driver documentation.

  2. Navigate to the clara-reference-workflows folder.

  3. Run the following command to generate the helm chart.

Copy
Copied!
            

./clara-wf cs <container-name>

For example, to generate a user-ai stage, run the following command:

Copy
Copied!
            

./clara-wf cs user-ai

That command generates [container-name] specific fields in the clara-reference-workflow/charts/clara-workflow/values.yaml file.

  1. In clara-reference-workflow/charts/clara-workflow/values.yaml, locate container-specific fields under the “stages:” section. Verify the key value pairs are same as defined by the user container. Most values can be used as they are. Update them manually, if needed.

  2. Pass container-specific arguments via workflow definition, if necessary.

For example, when adding the user-ai container as a stage with clara-wf, the following is generated for the user-ai container under the stage key in clara-reference-workflow/charts/clara-workflow/values.yaml:

Copy
Copied!
            

stages: ... ##BEGIN_user-ai## user-ai: image: repository: user-ai tag: latest mount: in: name: "input" out: name: "output" stageName: "user-ai" appDir: "/app" inputLock: "/app/locks/input.lock" inputs: "input" logName: "/app/logs/user-ai.log" outputs: "output" lockDir: "/app/locks" lockName: "user-ai.lock" timeout: "300" publishPath: "/publish" ##END_user-ai##


To Create a Workflow

To create a workflow named user-workflow with the dicom-reader container, the user-ai container and the dicom-writer container, perform the following steps. The dicom-reader and dicom-writer containers are provided in the Clara Deploy SDK. The user-ai container is a user-created AI container.

  1. Create and build the user-ai:latest container image with Clara Workflow Driver. The dicom-reader container image (clara/dicom-reader:[version]) and the dicom-writer container image (clara/dicom-writer:[version]) would be available once deployment script (deploy.sh) in Clara Deploy SDK is executed. For additional details see section 6.2 of the Clara Deploy SDK Documentation.

  2. Ensure that container specific Helm details are generated for each container and are present under the “stage:” key in clara-reference-workflow/charts/clara-workflow/values.yaml. Verify the correctness of each container specific key-value details.

  3. Generate the workflow ID. Run cat /proc/sys/kernel/random/uuid to create an ID that can be used for a new workflow.

#. Update clara-reference-workflow/charts/clara-workflow/values.yaml under the workflows: key with the newly created workflow. #.

Add the workflow ID, name (user-workflow), stages in the workflow, waitLocks for each stage, ioFolders for each stage, and args for any arguments, if desired, to be passed to the container(s).

For this example, the following configuration details can be used:

  • The workflow ID 7b413e24-d018-4689-8c41-73272f6649d4 is created.

  • The workflow name is user-workflow.

  • The workflow uses the dicom-reader, user-ai, and dicom-writer containers.

  • The sequence of containers is dicom-reader, then user-ai, then dicom-writer.

The Workflow starts with dicom-reader. The output of dicom-reader is fed to user-ai, and the output of user-ai is fed into dicom-writer. The user-ai container waits on the dicom-reader.lock file and dicom-writer waits on the user-ai.lock file. The ioFolders configuration specifies the input folder of each container. The last entry is an input to DICOM adapter to read the result.

Copy
Copied!
            

7b413e24-d018-4689-8c41-73272f6649d4: name: "user-workflow" stages: ["dicom-reader", "user-ai", "dicom-writer"] waitLocks: ["", "dicom-reader.lock", "user-ai.lock"] ioFolders: ["dicom-reader/input", "dicom-reader/output", "user-ai/ output", "dicom-writer/output"] args: ["", "", ""]

Copy
Copied!
            

If you want to provide arguments to the user-ai container's [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint) (e.g., `-p foo`), use the following configuration:

Copy
Copied!
            

args: ["", ["-p", "foo"], ""] Note: The random workflow ID can be used only for one workflow, for any other new workflow a new ID must be created.

  1. In clara-platform/files/dicom-server-config.yaml, update the following DICOM adaptor configuration settings.

    • Either use an existing ae-title or include the ae-title to be used under the ae-titles: key. If you add a new ae-title called user-ae-title, this must be configured with Nvidia.Clara.Dicom.Processors.JobProcessor,Nvidia.Clara.DicomServer in the Processors field. An existing ae-title can be used to configure the newly created ae-title.

    • Include the new workflow under the “workflows:” key with following details: name, clara-ae-title, destination name and workflow ID, such as the following:

Copy
Copied!
            

name: user-workflow clara-ae-title: CT_AI destination-name: DCM4CHEE workflow: 7b413e24-d018-4689-8c41-73272f6649d4

  1. To test the workflow locally without Clara Core and Clara DICOM adaptor, place input data of the workflow at “clara-reference-workflow/test-data/{random job ID in UUID}/{first stage name}/input” folder (e.g., clara-reference-workflow/test-data/11111111-1111-1111-1111-111111111111/dicom-reader/input), then execute the following command in the clara-reference-workflows directory:

Copy
Copied!
            

./clara-wf test [workflow-ID] [job-ID] [timeout in seconds (default:300)] For the example case, use the following command:

Copy
Copied!
            

./clara-wf test 7b413e24-d018-4689-8c41-73272f6649d4 11111111-1111-1111-1111-111111111111

  1. If you want to run the workflow with the deployed Clara, send DICOM data to DICOM adapter after copying TensorRT Inference Server model folder (if uesd) and publishing your workflow.

Copy
Copied!
            

# Copy model folder sudo cp -r [model folder path] /clara-io/models/ # Publish workflow sudo ./clara-wf publish 7b413e24-d018-4689-8c41-73272f6649d4 "User AI" /clara-io/clara-core/workflows/


Default Workflows

A set of default workflows are provided with this release. See clara-reference-workflow/charts/clara-workflow/values.yaml for details.

The Liver Segmentation workflow is one of the reference workflows provided with Clara Deploy SDK. This workflow takes an abdominal CT reconstructed volume (in DICOM format) as an input and processes the input volume with Liver Segmentation AI model. The AI model generates the labeled liver, and tumors (within the liver) as a binary mask (volume, same size as input). Liver is labeled as 1, tumors within the liver are labeled as 2, and the background is labeled as 0. Workflow in its final step, outputs the binary mask as a volume in DICOM format. The image below shows a slice of the input volumetric data (on left) in transverse direction and the segmented liver mask on the same slice (on right).

liverseg.jpg

Configuration Details

The Liver Segmentation workflow is defined as a reference workflow in clara-reference-workflows/charts/clara-workflow/values.yaml. The following are the configuration details for the workflow.

Copy
Copied!
            

fd3ee8bf-b9f3-4808-bd60-243f870ff9bd: name: "liver-seg" stages: ["dicom-reader", "ai-livertumor", "dicom-writer"] waitLocks: ["", "dicom-reader.lock", "ai-livertumor.lock"] ioFolders: ["dicom-reader/input", "dicom-reader/output", "ai-livertumor/output", "dicom-writer/output"] args: ["", "", ""]

  • Workflow ID: fd3ee8bf-b9f3-4808-bd60-243f870ff9bd

  • name: liver-seg

  • stages: “dicom-reader”, “ai-livertumor”, “dicom-writer”

  • waitlocks: “ai-livertumor” waits for dicom-reader.lock and “dicom-writer” waits for ai-livertumor.lock

  • ioFolders: defines data input folder for each container in the workflow. Last entry is the input to DICOM Adapter

  • args: No arguments are specified in the reference workflow

Helm Chart

The “ai-livertumor” helm chart is present in clara-reference-workflows/charts/clara-workflow/values.yaml under the “stages:” key.

Copy
Copied!
            

```bash ##BEGIN_ai-livertumor## ai-livertumor: image: repository: clara/ai-livertumor tag: [tag] mount: in: name: "input" out: name: "output" stageName: "ai-livertumor" appDir: "/app" inputLock: "/app/locks/input.lock" inputs: "input" logName: "/app/logs/ai-livertumor.log" outputs: "output" lockDir: "/app/locks" lockName: "ai-livertumor.lock" timeout: "300" publishPath: "/publish" ##END_ai-livertumor## ```

Where [tag] is the tag of the image for this release.

Workflow Execution

The Liver Segmentation workflow comes configured with DICOM Adapter in clara-platform/files/dicom-server-config.yaml. A default execution of the deployed Clara Deploy SDK executes the Liver Segmentation workflow. Optionally, the workflow can be executed using the clara-wf command line tool, with the following steps:

  1. Navigate to clara-reference-workflows folder

  2. Execute: ./clara-wf test fd3ee8bf-b9f3-4808-bd60-243f870ff9bd

© Copyright 2018-2019, NVIDIA Corporation. All rights reserved. Last updated on Feb 1, 2023.