Clara Application Container Developer Guide

This section describes how to develop an application to run in a Docker container on the NVIDIA Clara Deploy SDK, specifically an AI inference application. Also described are the steps to configure AI models and make them available to the NVIDIA TensorRT Inference Server, TRTIS.

The NVIDIA Clara Deploy SDK provides the Clara Pipeline Driver library. The library is written in the C programing language targeting the Linux platform, but also comes with bindings for C# and Python. In this section, the samples use the Python binding.

In this release, the Clara Deploy SDK Python binding is released in the form of a Python wheel, namely sdk_dist/clara-0.1-py3-none-any.whl. It defines a set of callback functions that an application must implement in order to integrate with the Clara Deploy SDK. The startup and cleanup functions are called at the beginning and end of the application life cycle, whereas prepare is called when a specific job is ready to be started, and execute is called when data is available for the specific job. For more details, see the Pipeline Driver library section.

The following are the event callbacks that must be implemented using the decorators:

  • @clara.startup_cb

  • @clara.prepare_cb

  • @clara.execute_cb

  • @clara.cleanup_cb

  • @clara.error_cb

The following sample code shows how these callback functions are implemented as global functions, though they can also be implemented as class functions.

main.py

Copy
Copied!
            

import clara @clara.startup_cb def startup(): print("[clara] startup() is called. Do app init as needed.") @clara.prepare_cb def prepare(): print("[clara] prepare() is called. Prepare for a new job.") @clara.execute_cb def execute(payload): print("[clara] execute() is called. Execute on input data and save output.") @clara.cleanup_cb def cleanup(): print("[clara] cleanup() is called. Clean up resources.") @clara.error_cb def error(result, message, entry_name, kind): print(result, message, entry_name, kind) if __name__ == "__main__": clara.start() # Wait until all callbacks are called. clara.wait_for_completion()

The execute callback accepts a payload parameter. You can get input/output file paths using code similar to the following:

Copy
Copied!
            

@clara.execute_cb def execute(payload): print("Input file paths:") for input_stream in payload.inputs: print(" - {}".format(input_stream.path)) print("Output file paths:") for output_stream in payload.outputs: print(" - {}".format(output_stream.path))

The Clara Deploy SDK Python wheel file, and the TRTIS client wheel file in the case of an AI inference application, need to be copied to your project, and installed in the Python virtual environment.

For samples of Python package structure, please refer to sections on reference pipeline containers.

The Clara Deploy SDK Pipeline Driver supports a testing mode enabled by the following environment variables:

Copy
Copied!
            

#!/bin/bash export NVIDIA_CLARA_RUNSIMPLE=TRUE export NVIDIA_CLARA_JOBID=$(cat /proc/sys/kernel/random/uuid) # Random uuid export NVIDIA_CLARA_JOBNAME="test-vnet" export NVIDIA_CLARA_STAGENAME="ai" export NVIDIA_CLARA_APPDIR=$(pwd) export NVIDIA_CLARA_INPUTS="input/recon.mhd" export NVIDIA_CLARA_OUTPUTS="output/recon.seg.mhd" export NVIDIA_CLARA_TRTISURI="localhost:8000" export NVIDIA_CLARA_PUBLISHPATH="publish"

With these environment variables set according to your local environment, the Pipeline Driver invokes the callbacks once your application starts. For example scripts to run tests locally, please see sections on reference pipeline containers.

Applications for Clara Deploy SDK must run in a Docker container, and should be tested in a development environment before being deployed on the Clara Deploy SDK.

For a sample script to test the Docker container in a development environment,

please see sections on reference pipeline containers.

The following is a sample Docker file:

Copy
Copied!
            

# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved. # # NVIDIA CORPORATION and its licensors retain all intellectual property # and proprietary rights in and to this software, related documentation # and any modifications thereto. Any use, reproduction, disclosure or # distribution of this software and related documentation without an express # license agreement from NVIDIA CORPORATION is strictly prohibited. FROM ubuntu:16.04 ENV PYVER=3.5 ENV LC_ALL=C.UTF-8 ENV LANG=C.UTF-8 # Use the same user name and home folder regardless of actual user. ENV USER=root ENV HOME=/root RUN apt-get update \ && apt-get install -y --no-install-recommends \ python3.5 \ curl \ libcurl3 \ rsync \ ca-certificates \ && curl -O https://bootstrap.pypa.io/get-pip.py \ && rm -rf /var/lib/apt/lists/* # Make /usr/bin/python point to the $PYVER version of python RUN rm -f /usr/bin/python \ && rm -f /usr/bin/python`echo $PYVER | cut -c1-1` \ && ln -s /usr/bin/python$PYVER /usr/bin/python \ && ln -s /usr/bin/python$PYVER /usr/bin/python`echo $PYVER | cut -c1-1` \ && python$PYVER get-pip.py \ && rm get-pip.py WORKDIR /app RUN mkdir -p input output # Copy code, scripts and packages COPY ./sdk_dist/*.whl ./sdk_dist/ COPY ./trtis_client/*.whl ./trtis_client/ COPY ./app_vnet ./app_vnet COPY ./Pipfile ./ # Remove unnecessary files in 3rd-party package (python3.5/site-packages/future/backports/test/*.pem) # Allow any users to access /root/.cache and /root/.local for pip/pipenv RUN pip install --upgrade setuptools pipenv \ && pipenv install --python 3.5 ./trtis_client/*.whl ./sdk_dist/*.whl --skip-lock \ && pipenv run pip install -r ./app_vnet/requirements.txt \ && rm -rf $(pipenv --venv)/lib/python3.5/site-packages/future/backports/test/*.pem \ && rm -rf /root/.cache/* \ && chmod -R 777 /root ENTRYPOINT ["pipenv", "run", "python", "app_vnet/main.py"]

The Clara Deploy SDK currently does not provide an API to upload an application Docker image into its own registry. An application Docker image needs to be copied

to the Clara Deploy SDK host, and then imported into the host’s Docker registry.

Please see Docker documentation on how to save and import a Docker image.

For an AI inference application, trained models need to be made available for the NVIDIA TensorRT Inference Server. For details on what models are supported, how to

configure a model, and the TRTIS model repository, please refer to the official

NVIDIA documentation at the following link: https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/.

After the model and configuration files are ready, they must be copied onto the Clara Deploy SDK, and saved in the shared storage folder that is mapped to the models folder of the TRTIS container, e.g. /clara-io/models.

Once the application Docker image is available on the Clara Deploy SDK, a pipeline

can be created to utilize it. Please refer to the section on pipeline creation

for details.

© Copyright 2018-2019, NVIDIA Corporation. All rights reserved. Last updated on Feb 1, 2023.