11.2. CT Reconstruction Pipeline

11.2.1. Overview

CT reconstruction pipeline is defined using the Pipeline definition language (that Clara supports). Sample reconstruction pipeline is located in clara-reference-app folder under SDK zip, Pipelines/ReconPipeline/ct-recon.yaml. The content is shown below,

api-version: 0.4.0
name: ct-recon-example
operators:
- name: recon-operator
  description: CT reconstruction algorithm on 3D cone beam projections
  container:
    image: clara/recon-operator
    tag: latest
  # Change the variables for the payload that is being used to run the pipeline
  variables:
    nvrtkalgo: fdk
    nvrtkhardware: gpu
    nvrtkhann: 0.95
    nvrtkdimension: 512,512,163
    nvrtkspacing: 0.75,0.75,3
    nvrtkorigin: -192,-192,-244.5
    nvrtkproj: 1024
    nvrtkindir: in
    nvrtkoutdir: out
    nvrtkgeomdir: geom
    nvrtklogs: logs
    nvrtksubset: 16
  requests:
    gpu: 1
  input:
  - path: /app/in
  output:
  - name: out
    path: /app/out
  - name: logs
    path: /app/logs
  - name: geom
    path: /app/geom

11.2.2. Data

Sample lung dataset in mhd format is available within the SDK. If sample dataset (recon_raw_proj_mhd_0.0.1.zip) is desired, it can be extracted from test-data folder under SDK zip. If using the sample dataset, unzip it before using.

11.2.3. Operators

Recon pipeline definition consists of 1 operator, recon-operator.

operators:
- name: recon-operator
  description: CT reconstruction algorithm on 3D cone beam projections
  container:
    image: clara/recon-operator
    tag: latest
  # Change the variables for the payload that is being used to run the pipeline
  variables:
    nvrtkalgo: fdk
    nvrtkhardware: gpu
    nvrtkhann: 0.95
    nvrtkdimension: 512,512,163
    nvrtkspacing: 0.75,0.75,3
    nvrtkorigin: -192,-192,-244.5
    nvrtkproj: 1024
    nvrtkindir: in
    nvrtkoutdir: out
    nvrtkgeomdir: geom
    nvrtklogs: logs
    nvrtksubset: 16
  requests:
    gpu: 1
  input:
  - path: /app/in
  output:
  - name: out
    path: /app/out
  - name: logs
    path: /app/logs
  - name: geom
    path: /app/geom
  • recon-operator: Recon operators takes the result from the Clara pipeline payload and mounts it to /app/in folder. Recon-operator has 3 output folders. ‘out’ folder gets the actual reconstructed volume. ‘logs’ folder gets all logs from recon-operator. ‘geom’ folder contains the geometry file created by recon-operator. Recon-operator defines the reconstruction parameters under ‘variables’. All parameters are defined in detail in recon operator definition. These variables are passed by platform as environment variables to the recon-operator at runtime. Reconstruction is GPU accelerated, and the need for GPU is specified with ‘request’ under recon-operator definition.

Please refer to the Run Reference Pipelines using Local Input Files in the How to run a Reference Pipeline section and the Pipeline Definition Language section for details on the pipeline definition language.

11.2.4. Execution of Reconstruction docker

11.2.4.1. Reconstruction Operator Docker Image

Pre-build reconstruction docker image is shipped with the SDK. All parameters within the reconstruction operator (as defined above) can be used by the user. Users can run reconstruction operator using the pipeline definition files and sample datasets. These are described below.

Before executing the reconstruction pipeline, ensure that all environment variables are correctly set and appropriate data folders are mounted. This can be done in the “variables” section of recon-operator within the pipeline definition file.

Get the data: Have projection data ready for execution. If sample dataset (recon_raw_proj_mhd_0.0.1.zip) is desired, it can be extracted from test-data folder under SDK zip. If using the sample dataset, unzip it before using.

11.2.5. Execute of Reconstruction Pipeline within Clara

Reconstruction pipeline definition is mandatory to start the process of running reconstruction on Clara platform. These pipelines are present in SDK zip under clara-reference-app folder.

11.2.5.1. Execution Steps on Clara Platform

The following are the execution steps for the CT Reconstruction Pipeline:

  • Locate the projection data /test-data/recon_raw_proj_mhd_0.0.1.zip. Unzip the file to a folder which creates a subfolder named ‘projections’.

    # cd <full path of clara installation folder>
    unzip test-data/recon_raw_proj_mhd_0.0.1.zip -d test-data/
    
  • Create a pipeline.

    # cd <full path of clara installation folder>
    clara create pipeline -p clara-reference-app/Pipelines/ReconPipeline/ct-recon.yaml
    
  • Verify the pipeline ID creation. Type the following command on the shell.

    kubectl get pl
    

    Pipeline with ID created in step 3 is visible in the kubectl pipelines listed.

  • Trigger the pipeline (push an input png file to Clara API Server) with the pipeline ID in the previous section.

    clara create jobs -n ct-recon -p <pipeline ID> -f test-data/projections/
    

    When the above step is completed, you should see the following result in the terminal (UUID of Job and Payload will be different).

    JOB_ID: daa5aea74d1446a8a24b9d3990bfef67
    PAYLOAD_ID: 79aa5500e6134cf888f1d6d89f2e249e
    

    Note

    Note down this job ID and payload ID as they would be used in the later section to access output files.

    Then, start the job:

    clara start job -j <JOB ID>
    

11.2.5.2. Check Job Status and Download/View Payloads in Clara Console/Dashboard

With Clara Orchestrator (API version >= 0.4.0)

  • Go to the Clara Management Console using a web browser: The URL is: <IP of the machine>:32002
  • Go to the JOBS view. You should see a job with a name that includes the name of the job you specified
  • You can inspect the status of each operator inside that job

Please check Clara Management Console to see the web application in detail.

With Argo Orchestrator (API version <= 0.3.0)

Go to the Clara dashboard UI using a web browser: The URL is <machine-ip>:8000.

  • You should see a job with a name that includes the name of the pipeline you created.
  • You can inspect the status of each operator inside that job

11.2.5.3. Download and Verify the Outputs

Once the job has completed successfully, we can download output payload by using clara download command.

Output payload files are located under /operators/<operator name defined in the pipeline definition>/ folder. Since we triggered CT Recon Pipeline whose operator’s name is recon-operator, we can use the following command to download output files:

mkdir result
cd result

clara download <Job ID>:/operators/recon-operator/*

Then, you can see the output MHD file at result/out folder.

If you are working with the server machine, the payload folder is directly accessible at the default payload folder /clara/payloads/<Payload ID>/.

11.2.6. CT Recon Perf pipeline

Pipeline ct-recon-perf.yaml is added. It runs iterative FDK reconstruction and uses 1024 projections (resulting in an input payload of 1.1 GB). With a realistic input payload, this pipeline can be used to analyze performance metrics. Number of iterations for the algorithm are set at 20, it can be increased to analyze a more realistic execution scenario.

11.2.7. CT Recon Liver segmentation pipeline

This pipeline performs the reconstruction on projections in MHD format, followed by liver ct segmentation. This pipeline takes MHD data as input.