7. How to Run a Reference Pipeline using Model Repository
Clara Deploy SDK comes bundled with a set of reference pipelines. This section provides general guidelines on how to execute them on your local development machine. For information specific to a reference pipeline, please view the documentation of that pipeline.
Change directory into a directory where pipelines will be stored, e.g.
mkdir -p ~/.clara/pipelines cd ~/.clara/pipelines
Use
clara pull pipeline
to download a pipeline definition:clara pull pipeline <NGC ID for the pipeline> cd <NGC ID for the pipeline>
For Example:
clara pull pipeline clara_ai_livertumor_pipeline cd clara_ai_livertumor_pipeline
Available pipelines are listed in the table below.
Pipeline Name | NGC ID | Model Type |
---|---|---|
AI Liver Tumor | clara_ai_livertumor_pipeline | tensorflow |
AI Brain Tumor | clara_ai_brain_tumor_pipeline | tensorflow |
AI Chest X-Ray Classification | clara_ai_chestxray_pipeline | tensorflow |
AI Colon Tumor Segmentation | clara_ai_colontumor_pipeline | tensorflow |
AI Spleen Segmentation | clara_ai_spleen_pipeline | tensorflow |
Unzip test data - if test data is bundled with the pipeline, run:
unzip <test-data-filename>.zip -d input
Unzip and Install model - if the model is bundled with the pipeline, run:
unzip <model-filename>.zip
Register the model with Model Repository
clara create model -p <path-to-model-dir>/ -t <tensorflow|tensorrt|pytorch|unknown>
Many reference pipelines are triggered by Clara DICOM Adaptor after it has received DICOM instances from external sender to one of its mapped AE Titles. This section explains the process with an example pipeline Liver Segmentation Pipeline.
- Navigate to the pipeline directory mentioned above. The YAML file defines the reference pipeline. The pipeline definition can be customized to suit a specific deployment environment.
7.2.1. External DICOM Sender and DICOM Receiver
You need an external DICOM Service Class User (SCU) application to send images to the Clara DICOM Adapter (acting as a DICOM SCP). Similarly, when your pipeline finishes executing, you may want to send the output to an external DICOM receiver.
For this example, we will use the open-source DICOM toolkit DCMTK to provide the external DICOM sender and DICOM receiver functionality.
7.2.1.1. Install DCMTK
Install DCMTK utilities by issuing the following command:
sudo apt-get install dcmtk
7.2.1.2. Setup an External DICOM Receiver
You can use the storescp utility as an external DICOM receiver. Open a terminal and run the following command to set up a DICOM Receiver. Create a directory in your machine where DICOM images produced by Clara Deploy SDK will be stored. We will refer to that directory as “dicom-destination-directory”
mkdir dicom-destination-directory
cd dicom-destination-directory
sudo storescp -v --fork -aet MYPACS 1004
MYPACS
is the Application Entity (AE) title of the external DICOM Receiver.
1004
is the port number where storescp
would be listening for any incoming DICOM traffic. Please note that the IP address will be the same as the local system.
7.2.1.3. Setup an External DICOM Sender
You can use the storescu utility from DCMTK as a DICOM sender.
7.2.2. Customize the Pipeline
Before publishing a pipeline, default settings in the pipeline definition YAML file can be modified to configure the
resource request and orchestration mode. We will use the liver-tumor-pipeline-model-repo.yaml
to show the steps.
7.2.2.1. Customize the Orchestration Mode
In this release of the Clara Deploy Platform, Clara Pipeline Driver [cpd] orchestration is supported, in addition to the Argo based orchestration mode.
This mode can be customized with the api-version
attribute in the pipeline definition YAML file, setting the api-version
to 0.3.0
makes the pipeline run in Argo mode, whereas setting this attribute to 0.4.0
Clara Pipeline Driver mode by default. The orchestration mode can also be explicitly set for api-version
0.4.0
and above, as shown in the following example liver-tumor-pipeline-model-repo.yaml
which specifies Clara Pipeline Driver mode.
api-version: 0.4.0
orchestrator: Clara
name: liver-tumor-pipeline-model-repo
7.2.3. Create the Pipeline
Creating a pipeline registers it with the platform. Once a pipeline is registered, it is represented with an ID. Follow the steps below:
- Open a terminal and navigate to the directory
clara_ai_livertumor_pipeline
which contains the reference pipeline
cd ~/.clara/pipelines/clara_ai_livertumor_pipeline
- Create the pipeline by issuing the following command
clara create pipeline -p liver-tumor-pipeline-model-repo.yaml
Clara CLI should successfully submit the pipeline and in return should print a pipeline ID.
Note down this pipeline ID as it would be used in a later section to define a new pipeline mapping in the DICOM adapter.
The command clara list pipelines
can be also used to list available pipelines.
- Verify that the pipeline is correctly stored by issuing the following command
clara list pipelines
- In the list of pipeline IDs, you should see the pipeline ID created in the previous step.
7.2.4. Configure the Clara DICOM Adapter
When you send images to the Clara Deploy SDK, the DICOM Adapter component acts as the internal DICOM Service Class Provider or a receiving agent.
Please refer to Clara DICOM Adapter for setup instructions.
7.2.5. Trigger Pipeline
A pipeline is triggered when a DICOM series is sent to the specific AE title of the Clara DICOM Adapter which is associated with that pipeline. To trigger this pipeline, you will need to have a set of relevant DICOM images that can be used as source input data.
For this pipeline, a sample test dataset in dicom format is available within the package in input/dcm
.
Open a terminal and go to the directory which contains all your DICOM images.
cd ~/.clara/pipelines/clara_ai_livertumor_pipeline/input/dcm
Trigger the pipeline (push DICOM data to the proper ae title)
storescu -v +sd +r -xb -aet "DCM4CHEE" -aec "LiverSeg" AA.BBB.CC.DD 104 ./
Replace “AA.BBB.CC.DD” with the IP of your local machine.
When the above step is completed, you should see the following result in the terminal
I: Received Store Response (Success) <br>
I: Releasing Association
7.2.6. Check Job Status and Download/View Payloads in Clara Console/Dashboard
With Clara Orchestrator (API version >= 0.4.0)
- Go to the Clara Management Console using a web browser:
The URL is:
<IP of the machine>:32002
- Go to the
JOBS
view. You should see a job with a name that includes the name of the job you specified - You can inspect the status of each operator inside that job
Please check Clara Management Console to see the web application in detail.
7.2.7. Verify That the External DICOM Receiver Received Your Images
If your pipeline outputs DICOM images and sends them to an external DICOM
receiver, you may want to verify that the images were indeed received. If you
used the storescp
utility from DCMTK as mentioned earlier, please use a terminal
to navigate to the “dicom-destination-directory” and verify whether the expected
images are stored there. Also, you may want to use a third-party DICOM
viewer to view those images.