7.2. Liver Tumor Segmentation Operator

7.2.1. Overview

This example is a containerized AI inference application, developed to be used as one of the operators in the Clara Deploy pipelines. The application and the Docker container are built based on the base AI application container, which provides the application framework to deploy Clara Train TLT trained models. The same execution configuration file, the same set of transform functions, and the same scanning window inference logic are used, however,

inference is performed on TensorRT Inference Server.

7.2.2. Inputs

This application, in the form of a Docker container, expects an input folder /input by default, which can be mapped to the host volume when the Docker container is started. Expected in this folder is a volume image file, of the format of Nifti or MetaImage. Further, it is expected that the volume image is constructed from a single series of a DICOM study, typically the axial series with data type of original primary.

7.2.3. Outputs

This application saves the segmentation results to an output folder, /output by default, which also can be mapped to a folder on the host volume. After the successful completion of the application, a segmentation volume image, of the format MetaImage, is saved in the output folder. The name of the output file is the same as that of the input file, due to certain limitations of the downstream consumer.

This example container also publishes data for the Clara Deploy Render Server, in the /publish folder by default. The original volume image, segmented volume image, along with a render config file are saved in this folder.

7.2.4. AI Model

The application uses the model, named segmentation_liver_v1, which uses the platform tensorflow_graphdef. The input tensor is of shape 96x96x96 with a single channel and the output is of the same shape with 3 channels.

The NVIDIA Clara Train Transfer Learning Toolkit, TLT, for Medical Imaging provides pre-trained models unique to medical imaging plus additional capabilities such as integration with the AI-assisted Annotation SDK for speeding up annotation of medical images. This allows the user to have access to AI assisted labeling [Reference].

This example application uses a model provided by the NVIDIA Clara Train TLT for liver tumor segmentation, namely segmentation_liver_v1, converted from TensorFlow Checkpoint model to tensorflow_graphdef using TLT model export tool. The input tensor is of shape 96x96x96 with a single channel and the output of the same shape with 3 channels.

This application also uses the same transforms library and the same configuration file for validation/inference pipeline during TLT model training. The key model attributes, e.g. the model name and network input dimensions, are saved in the model_config.json file and consumed by this application at runtime.

7.2.4.1. NVIDIA TensorRT Inference Server (TRTIS)

This application perform inference on the NVIDIA TensorRT Inference Server, TRTIS, which provides a cloud inferencing solution optimized for NVIDIA GPUs. The server provides an inference service via an HTTP or gRPC endpoint, allowing remote clients to request inferencing for any model being managed by the server. Read more on TRTIS

7.2.5. Directory Structure

The directories in the container are shown below. The application code is under /App and is from the base container, except for the files in the config directory, which are model specific. The trtis_client directory contains the TRTIS API client library, whereas the sdk_dist contains the Clara Train TLT transforms library. The medical directory contains compiled modules from Clara Train TLT and the writer directory contains a specialized writer which saves the segmentation result to a volume image file as MetaImage.

/app
└── app_base_inference
    ├── app.py
    ├── config
    │   ├── config.pbtxt
    │   ├── config_render.json
    │   ├── config_validation.json
    │   ├── __init__.py
    │   └── model_config.json
    ├── Dockerfile
    ├── executor.py
    ├── logging_config.json
    ├── main.py
    ├── medical
    ├── Pipfile
    ├── public
    │   └── docs
    │       └── README.md
    ├── writers
    │   ├── __init__.py
    │   ├── __init__.py
    │   ├── mhd_writer.py
    │   └── writer.py
    ├── trtis_client/
    └── sdk_dist/
/input
/output
/publish

7.2.6. Executing Operator Locally

If you want to see the internals of the container and want to manually run the application, follow these steps.

  1. Start the container in interactive mode. See the next section on how to run the container, and replace the docker run command with docker run --entrypoint /bin/bash

  2. Once in the the Docker terminal, ensure the current directory is /app.

  3. Type in command pipenv run python ./app_base_inference/main.py"

  4. Once finished, type exit.

7.2.7. Executing Operator in Docker

7.2.7.1. Prerequisites

  1. Ensure the Docker image of TRTIS has been imported into the local Docker repository by using the following command docker images and look for the image name ofTRTIS and the correct tag for the release, e.g. 19.02-py3.

  2. Ensure that the model folder, including the config.pbtxt, is present on the Clara Deploy host. Verify it by using the following steps:

    • Log on to the Clara Deploy host.

    • Check for the folder liver_segmentation_v1 under the directory /Clara-io/models.

7.2.8. Steps

7.2.8.1. Step 1

Change to your working directory, e.g. test_liver_tumor.

7.2.8.2. Step 2

Create, if they do not exist, the following directories under your working directory:

  • input containing the input image file.

  • output for the segmentation output.

  • publish for publishing data for the Render Server.

  • logs for the log files.

  • models models and copy over liver_segmentation_v1 folder.

7.2.8.3. Step 3

In your working directory, create a shell script, e.g. run_liver.sh or other name if you prefer, copy the sample content below, and save it.

#!/bin/bash

# Copyright (c) 2019, NVIDIA CORPORATION.  All rights reserved.
#
# NVIDIA CORPORATION and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA CORPORATION is strictly prohibited.

# Define the name of the app (aka operator), assumed same as the proj folder name
APP_NAME="app_livertumor"

# Define the TenseorRT Inference Server Docker image, which will be used for testing
# Use either local repo or NVIDIA repo
#TRTIS_IMAGE="clara/trtis"
TRTIS_IMAGE="nvcr.io/nvidia/tensorrtserver:19.02-py3"

# Launch the container with the following environment variables
# to provide runtime information.
export NVIDIA_CLARA_APPDIR="/app"
export NVIDIA_CLARA_TRTISURI="localhost:8000"

# Define the model name for use when launching TRTIS with only the specific model
MODEL_NAME="segmentation_liver_v1"

# Create Docker network so that containers can communicate on this network
NETWORK_NAME="container-demo"

# Create network
docker network create ${NETWORK_NAME}

# Run TRTIS(name: trtis), maping ./models/${MODEL_NAME} to /models/${MODEL_NAME}
# (localhost:8000 will be used)
nvidia-docker run --name trtis --network ${NETWORK_NAME} -d --rm --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -p 8000:8000 \
    -v $(pwd)/sampleData/models/${MODEL_NAME}:/models/${MODEL_NAME} ${TRTIS_IMAGE} \
    trtserver --model-store=/models


# Wait until TRTIS is ready
trtis_local_uri=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' trtis)
echo -n "Wait until TRTIS ${trtis_local_uri} is ready..."
while [ $(curl -s ${trtis_local_uri}:8000/api/status | grep -c SERVER_READY) -eq 0 ]; do
    sleep 1
    echo -n "."
done
echo "done"

export NVIDIA_CLARA_TRTISURI="${trtis_local_uri}:8000"

# Run ${APP_NAME} container.
# Launch the app container with the following environment variables internally,
# to provide input/output path information.
docker run --name ${APP_NAME} --network ${NETWORK_NAME} -it --rm \
    -v $(pwd)/input:/input \
    -v $(pwd)/output:/output \
    -v $(pwd)/logs:/logs \
    -v $(pwd)/publish:/publish \
    -e NVIDIA_CLARA_APPDIR \
    -e NVIDIA_CLARA_TRTISURI \
    ${APP_NAME}

echo "${APP_NAME} is done."

# Stop TRTIS container
echo "Stopping TRTIS"
docker stop trtis > /dev/null

# Remove network
docker network remove ${NETWORK_NAME} > /dev/null

7.2.8.4. Step 4

Execute the script below, and wait for the application container to finish, ./run_liver.sh.

7.2.8.5. Step 5

Check for the following output files:

  1. Segmentation results in the output directory

    • One file of the same name as your input file, with extension .mhd

    • One file of the same name, with extension .raw

  2. Published data in the publish directory

    • Original volume image, image.mhd and image.raw.

    • Segmentation volume image, image.seg.mhd and image.seg.raw.

    • Render Server config file, config_render.json. ### Step 6 To visualize the segmentation results, or the rendering on Clara Dashboard, please refer to Sections on visualization.