12.22. Clara Advanced DICOM Integration Pipeline

CAUTION: This is NOT for diagnostics use.

This asset requires the Clara Deploy SDK. Follow the instructions on the Clara Ansible page to install the Clara Deploy SDK.

12.22.1. Overview

This pipeline is one of the reference pipelines provided with Clara Deploy SDK, showcaseing the advanced integration of AI inference with DICOM image input and output, highlighted below:

  • Use whole study as input to the pipeline
  • Use DICOM Parser to select specific series by ways of configuring the well-known environment variable with series selection rules in Base64 encoded text. This allows selecting a series and its volume for inference, or multiple series and their images for the image registration operator that combines the images into a single multi-channel volume image
  • Use the specific DICOM IOD writers to create DICOM instances for the segmentation image out of AI inference, e.g. DICOM Segmentation and DICOM RT Structure Set.

A pre-trained model for volumetric (3D) brain tumor segmentation (only tumor core (TC) from T1c images) is used in the pipeline, hence the AI operator takes in as input a single-channel image converted from the T1c series of a MRI DICOM study. The model generates the labeled tumor cores within the brain as a binary mask on slices of the volume at the same size as the input. The brain tumor core is labeled as 1, and the background is labeled as 0.

For executing of this pipeline, the Clara Deploy DICOM Adapter is pushed the DICOM instances, even with all series in the MRI study. Once the DICOM instances are all received, the pipeline is started to first convert the applicalbe DICOM series to volume images in NIfTI, or MetaIO format per configuration, and importantly, the configured series selecton rules are applied to select the specific series to complete the processing, as described in the beginning paragraph of this section.

In its final steps, the pipeline outputs the binary mask as a new DICOM Segmentation series using the original study instance UID, as well as the DICOM RT Structure Set series just for demonstration. In addition, the original and segmented volumes are published to the Clara Deploy Render Server for visualization on the Clara Console.

12.22.2. Pipeline Definition

This pipeline is defined in the Clara Deploy pipeline definition language, and utilizes built-in reference containers to construct the following set of processing stages:

  • The dicom-parser operator converts input DICOM data into volume images in NIfTI format, saves the metadata file mapping the series to image files, selects the specific series with the configured selection rules, and finally saves the metadata file mapping selection name to series to image file
  • The brain-tumor-segmentation operator performs AI inference against the NVIDIA Triton Inference server to generate brain- and tumor-segmentation volume images
  • The dicom-seg-writer converts the segmented volume image into DICOM Seg series, with a new series instance UID and the original study instance UID. It consumes the metadata of the selection name and selected series to retrieve the DICOM metadata from the specific series
  • The register-dicom-output operator registers the DICOM instances with the Clara Deploy DICOM Adapter, which in turn stores each instance on external DICOM devices per the configuration
  • The rtstruct-writer performs similarly to dicom-seg-writer in paralell
  • The register-volume-images-for-rendering operator registers original and segmented volume images with the Clara Deploy Render Server for visualization

The following are details of the pipeline definition, with comments describing how each operator functions, as well as their input and output.

api-version: 0.4.0
name: advanced-dicom-integration-pipeline
operators:
# DICOM parser operator
# This operator parses DICOM instances to generate images for applicable series,
# e.g. CT/MR/XRay. It can also be configured with series selection rules,
# so that only specific series and its image will be selected and the information
# is stored in a well-known metadata file. This allows the whole study to be sent
# to the pipeline even if the downstream operators, e.g. the inference operator,
# can only consume only one image converted from a single series.
# In this example, the selection rules are Base64 encoded as required and assigned
# to the well-known environment variable, NVIDIA_CLARA_SELECTION_RULES_BASE64
#
# Input: '/input' mapped directly to the input of the pipeline, i.e. DICOM instances.
# Output:'/output' for saving converted volume image in MHD format to file whose name
#            is the same as the DICOM series instance ID, a series to image metadata
#            file, and a selection name to series to image mapping metadata file.
- name: dicom-parser
  description: Converts DICOM instances into image files, one per DICOM series, along with metadata files.
  container:
    image: clara/dicom-parser
    tag: latest
  variables:
    NVIDIA_CLARA_SELECTION_RULES_BASE64: ewogICAgInNlbGVjdGlvbnMiOiBbCiAgICAgICAgewogICAgICAgICAgICAibmFtZSI6ICJUMUMiLAogICAgICAgICAgICAiY29uZGl0aW9ucyI6IHsKICAgICAgICAgICAgICAgICJNb2RhbGl0eSI6ICIoP2kpXk1SIiwKICAgICAgICAgICAgICAgICJTdHVkeURlc2NyaXB0aW9uIjogIig/aSlCcmFpbiIsCiAgICAgICAgICAgICAgICAiSW1hZ2VUeXBlIjogWwogICAgICAgICAgICAgICAgICAgICAgICAgICAgIk9SSUdJTkFMIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICJQUklNQVJZIgogICAgICAgICAgICAgICAgICAgICAgICBdLAogICAgICAgICAgICAgICAgIlNlcmllc0Rlc2NyaXB0aW9uIjogIl4oPz0uKlQxYykoPyEuKihyZWZ8cmZtdHxbMC05XXhbMC05XSkpLioiCiAgICAgICAgICAgIH0KICAgICAgICB9CiAgICBdCn0=
  input:
  - path: /input
  output:
  - path: /output
# brain-tumor-segmentation operator
# Input: `/input` containing volume image data, MHD format, with a single volume.
# Output: `/output` containing segmented volume image, MHD format.
#         `/publish` containing original and segmented volume images, MHD format,
#             along with rendering configuration file.
- name: brain-tumor-segmentation
  description: Segmentation of brain tumor core inferencing using DL trained model.
  container:
    image: clara/ai-braintumor
    tag: latest
  requests:
    memory: 8192
  input:
  - from: dicom-parser
    path: /input
  output:
  - path: /output
    name: segmentation
  - path: /publish
    name: rendering
  services:
  - name: trtis
  # Triton Inference Server, required by this AI application.
    container:
      image: nvcr.io/nvidia/tritonserver
      tag: 20.07-v1-py3
      command: ["tritonserver", "--model-repository=$(NVIDIA_CLARA_SERVICE_DATA_PATH)/models"]
    # services::connections defines how the TRTIS service is expected to
    # be accessed. Clara Platform supports network ("http") and
    # volume ("file") connections.
    connections:
      http:
      # The name of the connection is used to populate an environment
      # variable inside the operator's container during execution.
      # This AI application inside the container needs to read this variable to
      # know the IP and port of TRTIS in order to connect to the service.
      - name: NVIDIA_CLARA_TRTISURI
        port: 8000
      # Some services need a specialized or minimal set of hardware. In this case
      # NVIDIA Tensor RT Inference Server [TRTIS] requires at least one GPU to function.
# dicom segmentation writer operator
# Input1: `/input` containing a volume image file, in MHD format, name matching the DICOM series instance UID.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm file.
# Input3: `/series_selection` contains output from DICOM Parser, specifically for the `selected-images.json` file.
# Output: `/output` containing the DICOM instances converted from the volume image, with updated attributes
#         based on original DICOM instances.
- name: dicom-seg-writer
  description: Converts MHD into DICOM instance with attributes based on the original instances.
  container:
    image: clara/dicom-seg-writer
    tag: latest
  variables:
    NVIDIA_SEG_LABELS: "[\"Tumor-seg\"]"
  input:
  - from: brain-tumor-segmentation
    name: segmentation
    path: /input
  - from: dicom-parser
    path: /series_selection
  - path: /dcm
  output:
  - path: /output
    name: dcm
  - path: /logs
    name: logs
# register-volume-images-for-rendering operator
# Input: Published original and segmented volume images, MHD format, along with rendering configuration file
#        from the segmentation operator.
# Output: N/A. Input data will be sent to the destination, namely `renderserver` for Render Server DataSet Service.
- name: register-volume-images-for-rendering
  description: Register volume images for rendering.
  container:
    image: clara/register-results
    tag: latest
    command: ["python", "register.py", "--agent", "renderserver"]
  input:
  - from: brain-tumor-segmentation
    name: rendering
    path: /input
# register-dicom-output operator
# Input: `/input` containing DICOM instances in the named output, `dicom` from dicom-seg-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-dicom-output
  description: Register converted DICOM instances with Results Service to be sent to external DICOM devices.
  container:
    image: clara/register-results
    tag: latest
    command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
  input:
  - from: dicom-seg-writer
    name: dcm
    path: /input
# dicom RTSTRUCT writer operator
# Input1: `/input` containing a volume image file, in MHD format.
# Input2: `/dcm` containing original DICOM instances, i.e. dcm files.
# Input3: `/series_selection` contains output from DICOM Parser, specifically for the `selected-images.json` file.
# Output: `/output` containing the DICOM RT Structure Set instance for the segmentation mask contours,
#         with attributes referring to the original DICOM Series.
- name: rtstruct-writer
  description: Converts MHD into DICOM RTSTRUCT instance with attributes rfererring to the original instances.
  container:
    image: clara/rtstruct_writer
    tag: latest
  requests:
    memory: 2048
  input:
  - from: brain-tumor-segmentation
    name: segmentation
    path: /input
  - from: dicom-parser
    path: /series_selection
  - path: /dcm
  output:
  - path: /output
    name: dcm
  - path: /logs
    name: logs
# register-dicom-RTSTRUCT-output operator
# Input: `/input` containing DICOM instance in the named output, `dicom` from rtstruct-writer operator.
# Output: N/A. Input data will be sent to the destinations, namely DICOM devices, by the Clara DICOM SCU agent.
- name: register-dicom-rtstruct-output
  description: Register converted DICOM RTSTRUCT instance with Results Service to be sent to external DICOM devices.
  container:
    image: clara/register-results
    tag: latest
    command: ["python", "register.py", "--agent", "ClaraSCU", "--data", "[\"MYPACS\"]"]
  input:
  - from: rtstruct-writer
    name: dcm
    path: /input

12.22.3. Executing the Pipeline

Please refer to the Quick Start Guide on how to register a pipeline, configure the DICOM Adapter, and execute the pipeline.

12.22.4. License

An End User License Agreement is included with the product. By pulling and using the Clara Deploy asset on NGC, you accept the terms and conditions of these licenses.

12.22.5. Suggested Reading

Release Notes, the Getting Started Guide, and the SDK itself are available at the NVIDIA Developer forum.

For answers to any questions you may have about this release, visit the NVIDIA Devtalk forum.