12.21. Clara Deploy AI COVID-19 Classification Pipeline
CAUTION: This is NOT for diagnostics use.
This asset requires the Clara Deploy SDK. Follow the instructions on the Clara Ansible page to install the Clara Deploy SDK.
This inference pipeline was developed by NVIDIA. It is based on a segmentation and classification model developed by NVIDIA researchers in conjunction with the NIH. The Software is for Research Use Only. Software’s recommendation should not be solely or primarily relied upon to diagnose or treat COVID-19 by a Healthcare Professional. This research use only software has not been cleared or approved by FDA or any regulatory agency
This reference pipeline is designed to infer the probability of COVID-19 infection with the chest CT scan of a patient. A 3D lung segmentation model as well as a 3D classification model are used in this pipeline. A single aixial DICOM series of chest CT scan is the pipeline input, and the final output is the report of the probabilities of COVID-19 and non-COVID-19.
Once the DICOM instances are received, the pipeline is triggered to first convert the DICOM instances to a volume image in MetaImage format. This image is then used as the input to two segmentation operators concurrently, the lugn segmentation and the COVID-19 lesion segmentation. Each segmentation operator performs inference using its own segmentation AI model, and generates a labeled segmentation as a binary mask on each slice of the volume, with the lung or lung lesion labeled as 1, and the background labeled as 0.
In the next step, the original volume image along with the labeled segmentation image are used by the COVID-19 classification operator to infer the probabilities of COVID-19 using the COVID-19 classification model.
The volume images generated by the COVID-19 lesion segmentation operator and the lung segmentation operator are also used to as inputs to a volume ratio metrics operator, which calculates and saves the lesion to lung ratio.
The pipeline generates the following outputs:
- Lung segmentation image in MetaImage format
- COVID-19 lesion segmentation image in MetaImage format
- A new DICOM series for the lung segmentation image, optionally sent to a DICOM device
- A new DICOM series for the COVID-19 lesion segmentation image, optionally sent to a DICOM device
- A new DICOM series of Encapsulated PDF for the AI classification results, optionally sent to a DICOM device
- A new DICOM series of Encapsulated PDF for the COVID-19 lesion to lung ratio results, optionally sent to a DICOM device
- The original and lung segmentation volumes in MetaImage format to the Clara Deploy Render Server.
- The original and COVID-19 lesion segmentation volumes in MetaImage format to the Clara Deploy Render Server.
This pipeline is defined in the Clara Deploy pipeline definition language. It utilizes built-in reference containers to construct the following set of operators:
- The dicom-reader operator converts input DICOM data into volume images in MetaImage format format.
- The lung segmentation operator performs AI inference against the NVIDIA Triton Inference Server to generate segmentation volume images.
- The COVID-19 lesion segmentation operator performs AI inference against the NVIDIA Triton Inference Server to generate segmentation volume images.
- The classification operator performs AI inference against the NVIDIA Triton Inference Server to infer the probability of COVID-19.
- The COVID-19 lesion to lung volume ratio operator.
- The dicom-seg-writer converts the lung segmentation volume image into a DICOM SEG instance with a new series instance UID but the same study instance UID of the original DICOM.
- The register-dicom-output operator registers the lung DICOM SEG instance with the Clara Deploy DICOM Adapter which in turn stores the instance on external DICOM devices per configuration.
- The dicom-seg-writer for COVID-19 lesion converts the COVID-19 lesion segmentation volume image into a DICOM SEG instance with a new series instance UID but the same study instance UID of the original DICOM.
- The register-dicom-output operator for COVID-19 lesion registers the lesion DICOM SEG instance with the Clara Deploy DICOM Adapter which in turn stores the instance on external DICOM devices per configuration.
- The dicom-report-writer converts the AI classification results into DICOM Encapsulated PDF instances with a new series instance UID but the same study instance UID of the original DICOM.
- The dicom-report-writer for volume ratio metrics converts the COVID-19 lesion to lung volume ratio metrics into a DICOM Encapsulated PDF instances with a new series instance UID but the same study instance UID of the original DICOM.
- The register-dicom-report operator registers the DICOM Encapsulated PDF with the Clara Deploy DICOM Adapter which in turn stores the instance on external DICOM devices per configuration.
- The register-volume-images-for-rendering operator registers original and segmented volume images with the Clara Deploy Render Server for visualization.
The following is the details of pipeline definition, with comments describing each operator’s functions as well as input and output.
api-version: 0.4.0
name: COVID-19-pipeline
operators:
# dicom reader operator
# Input: '/input' mapped directly to the input of the pipeline, which is populated by the DICOM Adaptor.
# Output:'/output' for saving converted volume image in MHD format to file whose name
# is the same as the DICOM series instance ID.
- name: dicom-reader
description: Converts DICOM instances into MHD, one file per DICOM series.
container:
image: clara/dicom-reader
tag: latest
input:
- path: /input
output:
- path: /output
# lung-segmentation operator
# Input: `/input` containing volume image data, MHD format, with a single volume.
# Output: `/output` containing segmented volume image, MHD format.
# `/publish` containing original and segmented volume images, MHD format,
# along with rendering configuration file.
- name: lung-segmentation
description: Segmentation of lung using DL trained model.
container:
image: clara/ai-lung
tag: latest
requests:
memory: 8192
input:
- from: dicom-reader
path: /input
output:
- path: /output
name: segmentation
- path: /publish
name: rendering
services:
- name: triton
# Triton Inference Server, required by this AI application.
container:
image: nvcr.io/nvidia/tritonserver
tag: 20.07-v1-py3
command: ["tritonserver", "--model-repository=$(NVIDIA_CLARA_SERVICE_DATA_PATH)/models"]
# services::connections defines how the Triton service is expected to
# be accessed. Clara Platform supports network ("http") and
# volume ("file") connections.
connections:
http:
# The name of the connection is used to populate an environment
# variable inside the operator's container during execution.
# This AI application inside the container needs to read this variable to
# know the IP and port of Triton in order to connect to the service.
- name: NVIDIA_CLARA_TRTISURI
port: 8000
# Some services need a specialized or minimal set of hardware. In this case
# NVIDIA Triton Inference Server [Triton] requires at least one GPU to function.
# dicom segmentation writer operator
# Input1: `/input` containing a volume image file, in MHD format, name matching the DICOM series instance UID.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm file.
# Output: `/output` containing the DICOM instances converted from the volume image, with updated attributes
# based on original DICOM instances.
- name: dicom-seg-writer
description: Converts MHD into DICOM instance with attributes based on the original instances.
container:
image: clara/dicom-seg-writer
tag: latest
variables:
NVIDIA_SEG_LABELS: "[\"Lung-seg\"]"
input:
- from: lung-segmentation
name: segmentation
path: /input
- path: /dcm
output:
- path: /output
name: dicom
- path: /logs
name: logs
# register-volume-images-for-rendering operator
# Input: Published original and segmented volume images, MHD format, along with rendering configuration file
# from the segmentation operator.
# Output: N/A. Input data will be sent to the destination, namely `renderserver` for Render Server DataSet Service.
- name: register-volume-images-for-rendering
description: Register volume images, MHD format, for rendering.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "renderserver"]
input:
- from: lung-segmentation
name: rendering
path: /input
# register-dicom-output operator
# Input: `/input` containing DICOM instances in the named output, `dicom` from dicom-seg-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-dicom-output
description: Register converted DICOM instances with Results Service to be sent to external DICOM devices.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
input:
- from: dicom-seg-writer
name: dicom
path: /input
## Begin: COVID-19 Lesion
# covid-lesion-segmentation operator
# Input: `/input` containing volume image data, MHD format, with a single volume.
# Output: `/output` containing segmented volume image, MHD format.
# `/publish` containing original and segmented volume images, MHD format,
# along with rendering configuration file.
- name: covid-lesion-segmentation
description: Segmentation of COVID-19 GGO using DL trained model.
container:
image: clara/ai-covid-lesion
tag: latest
requests:
memory: 8192
input:
- from: dicom-reader
path: /input
output:
- path: /output
name: segmentation
- path: /publish
name: rendering
services:
- name: triton
# Triton Inference Server, required by this AI application.
container:
image: nvcr.io/nvidia/tritonserver
tag: 20.07-v1-py3
command: ["tritonserver", "--model-repository=$(NVIDIA_CLARA_SERVICE_DATA_PATH)/models"]
# services::connections defines how the Triton service is expected to
# be accessed. Clara Platform supports network ("http") and
# volume ("file") connections.
connections:
http:
# The name of the connection is used to populate an environment
# variable inside the operator's container during execution.
# This AI application inside the container needs to read this variable to
# know the IP and port of Triton in order to connect to the service.
- name: NVIDIA_CLARA_TRTISURI
port: 8000
# Some services need a specialized or minimal set of hardware. In this case
# NVIDIA Triton Inference Server [Triton] requires at least one GPU to function.
# dicom covid-lesion segmentation writer operator
# Input1: `/input` containing a volume image file, in MHD format, name matching the DICOM series instance UID.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm file.
# Output: `/output` containing the DICOM instances converted from the volume image, with updated attributes
# based on original DICOM instances.
- name: dicom-seg-writer-covid-lesion
description: Converts MHD into DICOM instance with attributes based on the original instances.
container:
image: clara/dicom-seg-writer
tag: latest
variables:
NVIDIA_SEG_LABELS: "[\"GroundGlass-seg\"]"
input:
- from: covid-lesion-segmentation
name: segmentation
path: /input
- path: /dcm
output:
- path: /output
name: dicom
- path: /logs
name: logs
# register-volume-images-for-rendering operator
# Input: Published original and segmented volume images, MHD format, along with rendering configuration file
# from the segmentation operator.
# Output: N/A. Input data will be sent to the destination, namely `renderserver` for Render Server DataSet Service.
- name: register-covid-lesion-volume-images-for-rendering
description: Register volume images, MHD format, for rendering.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "renderserver"]
input:
- from: covid-lesion-segmentation
name: rendering
path: /input
# register-dicom-output operator
# Input: `/input` containing DICOM instances in the named output, `dicom` from dicom-seg-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-covid-lesion-dicom-output
description: Register converted DICOM instances with Results Service to be sent to external DICOM devices.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
input:
- from: dicom-seg-writer-covid-lesion
name: dicom
path: /input
## END: COVID-19 lesion segmentation
# COVID-19 Classification operator
# Input: original image (DICOM series converted image) and segmented volume images, MHD format.
# Output: CSV file for classification resuls: the probabilities for both COVID-19 and non-COVID-19.
- name: classification-covid-19
description: Classification of COVID-19 using DL model with original and segmentation images.
container:
image: clara/ai-covid-19
tag: latest
requests:
memory: 4096
input:
- from: lung-segmentation
name: segmentation
path: /label_image
- from: dicom-reader
path: /input
output:
- path: /output
name: classification
services:
- name: triton
# Triton Inference Server, required by this AI application.
container:
image: nvcr.io/nvidia/tritonserver
tag: 20.07-v1-py3
command: ["tritonserver", "--model-repository=$(NVIDIA_CLARA_SERVICE_DATA_PATH)/models"]
# services::connections defines how the Triton service is expected to
# be accessed. Clara Platform supports network ("http") and
# volume ("file") connections.
connections:
http:
# The name of the connection is used to populate an environment
# variable inside the operator's container during execution.
# This AI application inside the container needs to read this variable to
# know the IP and port of Triton in order to connect to the service.
- name: NVIDIA_CLARA_TRTISURI
port: 8000
# Some services need a specialized or minimal set of hardware. In this case
# NVIDIA Triton Inference Server [Triton] requires at least one GPU to function.
# dicom report writer operator
# Input1: `/input` containing a classification result file, in txt or csv format.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm file.
# Output: `/output` containing the DICOM report instances converted with attributes
# from the original DICOM instances.
- name: dicom-report-writer
description: Create DICOM AI Results report, with attributes based on the original instances.
container:
image: clara/dicom-report-writer
tag: latest
variables:
NVIDIA_DICOM_REPORT_TYPE: 'pdf'
NVIDIA_AI_MODEL_CREATOR: 'NVIDIA/NIH'
NVIDIA_AI_MODEL_NAME: 'ClaraCTClassificationCOVID-19'
NVIDIA_AI_MODEL_VERSION: '1.0'
input:
- from: classification-covid-19
name: classification
path: /input
- path: /dcm
output:
- path: /output
name: dicom
- path: /logs
name: logs
# register-dicom-report operator
# Input: `/input` containing DICOM instances in the named output, `dicom` from dicom-seg-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-dicom-report
description: Register DICOM instances with Results Service to be sent to external DICOM devices.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
input:
- from: dicom-report-writer
name: dicom
path: /input
## Begin: covid_lung_ratio
# COVID-19 Classification operator
# Input: COVID-19 lesion volume image and segmented lung volume images, MHD format.
# Output: txt file for volume ratio metrics.
- name: covid-lung-volume-ratio
description: Classification of COVID-19 using DL model with original and segmentation images.
container:
image: clara/ai-volume-ratio
tag: latest
requests:
memory: 2048
input:
- from: lung-segmentation
name: segmentation
path: /label_image
- from: covid-lesion-segmentation
name: segmentation
path: /input
output:
- path: /output
name: metrics
# dicom report writer operator
# Input1: `/input` containing a classification result file, in txt or csv format.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm file.
# Output: `/output` containing the DICOM report instances converted with attributes
# from the original DICOM instances.
- name: dicom-report-writer-metrics
description: Create DICOM AI Results report, with attributes based on the original instances.
container:
image: clara/dicom-report-writer
tag: latest
variables:
NVIDIA_DICOM_REPORT_TYPE: 'pdf'
NVIDIA_AI_MODEL_CREATOR: 'NVIDIA/NIH'
NVIDIA_AI_MODEL_NAME: 'ClaraCOVID-19LungLesion'
NVIDIA_AI_MODEL_VERSION: '1.0'
input:
- from: covid-lung-volume-ratio
name: metrics
path: /input
- path: /dcm
output:
- path: /output
name: dicom
- path: /logs
name: logs
# register-dicom-report operator
# Input: `/input` containing DICOM instances in the named output, `dicom` from dicom-seg-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-dicom-report-metrics
description: Register DICOM instances with Results Service to be sent to external DICOM devices.
container:
image: clara/register-results
tag: latest
command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
input:
- from: dicom-report-writer-metrics
name: dicom
path: /input
## End covid_lung_ratio
Please refer to the Quick Start Guide section on how to register a pipeline, configure the DICOM Adapter, and execute the pipeline.
An End User License Agreement is included with the product. By pulling and using the Clara Deploy asset on NGC, you accept the terms and conditions of these licenses. For Clara Deploy AI COVID-19 Classification Pipeline you accept the terms and conditions that are mentioned in the license file inside the package.
Release Notes, the Getting Started Guide, and the SDK itself are available at the NVIDIA Developer forum.
For answers to any questions you may have about this release, visit the NVIDIA Devtalk forum.