Advanced Usage#

Multi-GPU Load Balancing#

VISTA-3D NIM supports Multi-GPU load balancing using Triton Inference Server. This feature allows you to distribute inference tasks across multiple GPUs, potentially improving throughput and performance for high-volume workloads. However, this functionality is optional and not required for basic operation.

To enable Multi-GPU support:

  1. Ensure your system has multiple compatible GPUs.

  2. When launching the Docker container, specify the GPUs you want to use by setting the CUDA_VISIBLE_DEVICES environment variable.

  3. When launching the Docker container, set the NIM_HTTP_MAX_WORKERS environment variable to specify the number of workers for handling HTTP requests. Ensure this number is at least equal to the number of GPUs.

docker run --gpus all --rm -it --ipc=host --net=host -e NGC_API_KEY=$NGC_API_KEY -e CUDA_VISIBLE_DEVICES=0,1,2,3 -e NIM_HTTP_MAX_WORKERS=4 nvcr.io/nim/nvidia/vista3d:1.0.0

Infer with Local Image Files or DICOM Folder#

In addition to sending an image URL, you can also specify a local image file (or a folder that contains multiple image slices, such as a dicom folder) for inference requests. To enable this feature, the relevant path must be mounted when running the NIM container:

docker run --gpus all --rm -it --ipc=host --net=host -e NGC_API_KEY=$NGC_API_KEY -v /path/on/host:/path/in/container nvcr.io/nim/nvidia/vista3d:1.0.0

If the image file is /path/on/host/image.nii.gz, in the inference request payload, “image” should be /path/in/container/image.nii.gz. If the image folder is /path/on/host/dicom_folder, in the inference request payload, “image” should be /path/in/container/dicom_folder.

Writing Data to Local Mounted Directory#

To write inference results to a local mounted directory, the local_working_dir parameter should be specified in the request payload. To enable this feature, the local directory must be mounted when running the NIM container.

please ensure that the directory has the correct permissions to allow the Docker container to write to it (for example, do sudo chmod -R 777 to it). If the directory is /path/on/host/results/, then in the inference request payload, local_working_dir should be /path/in/container/results/.

Override Bundle#

If a bundle needs to be overridden, please provide an environment variable OVERRIDE_DICT when running the container. The dict should be a JSON-encoded string. For example:

docker run --gpus all --rm -it --ipc=host --net=host -e NGC_API_KEY=$NGC_API_KEY -e OVERRIDE_DICT='{"output_ext": ".nrrd"}' nvcr.io/nim/nvidia/vista3d:1.0.0

Infer without TensorRT#

By default, TensorRT is used for inference to leverage its performance optimizations. However, there may be scenarios where you want to disable TensorRT and use the default inference engine instead. To disable TensorRT, set the environment variable TRT_INFERENCE to False when running the container:

docker run --gpus all --rm -it --ipc=host --net=host -e NGC_API_KEY=$NGC_API_KEY -e TRT_INFERENCE=False <vista3d nim container>

Change Port#

The default port for the FastAPI app is 8000, and adding an environment variable NIM_HTTP_API_PORT when running the container can change it. For example:

docker run --gpus all --rm -it --ipc=host --net=host -e NGC_API_KEY=$NGC_API_KEY -e NIM_HTTP_API_PORT=12000 nvcr.io/nim/nvidia/vista3d:1.0.0

Note: Some ports, such as 8001, 8002, and 8080, may already be in use by other services inside the container. Avoid setting NIM_HTTP_API_PORT to these ports to prevent conflicts.

Image URL Restrictions#

To ensure the security and integrity of the VISTA-3D microservice, certain restrictions are applied to the URLs from which images can be fetched. These restrictions can be customized through environment variables, allowing for flexibility in different deployment scenarios.

Environment Variables#

  • ALLOWED_MIME_TYPES: Defines the MIME types that are allowed for image URLs.

    • Default: ["application/dicom"]

    • Example: -e ALLOWED_MIME_TYPES='["application/dicom"]'

  • DOMAIN_WHITELIST: Specifies a list of domains from which images can be fetched.

    • Default: ["https://.*", "https://raw.githubusercontent.com/NVIDIA/.*", "https://assets.ngc.nvidia.com/products/api-catalog/vista3d/.*", "https://storage.googleapis.com/.*", "https://.*.s3.amazonaws.com/.*", "https://.*.blob.core.windows.net/.*"]

    • Example: -e DOMAIN_WHITELIST='["https://example.com", "https://trusted-domain.org"]'

  • DOMAIN_BLACKLIST: Specifies a list of domains from which images cannot be fetched.

    • Default: []

    • Example: -e DOMAIN_BLACKLIST='["https://untrusted-domain.com", "https://malicious-site.org"]'

  • IMAGE_CONNECTION_TIMEOUT: Sets the timeout in seconds for connecting to an image URL.

    • Default: 10 seconds

    • Example: -e IMAGE_CONNECTION_TIMEOUT=15

  • IMAGE_READ_TIMEOUT: Sets the timeout in seconds for reading an image from a URL.

    • Default: 60 seconds

    • Example: -e IMAGE_READ_TIMEOUT=90

  • IGNORE_SSL_ERRORS: If set to True, SSL errors will be ignored when fetching images.

    • Default: False

    • Example: -e IGNORE_SSL_ERRORS=True

  • MAX_IMAGE_SIZE: Defines the maximum size in bytes for an image that can be processed.

    • Default: 1 GB (~1000000000 bytes)

    • Example: -e MAX_IMAGE_SIZE=2147483648 # 2 GB

  • DOWNLOAD_CHUNK_SIZE: Defines the size of each chunk in bytes when downloading large files.

    • Default: 10 MB (~10000000 bytes)

    • Example: -e DOWNLOAD_CHUNK_SIZE=16777216 # 16 MB

  • IMAGE_URI_HTTPS_ONLY: When set to True, only HTTPS URLs are allowed for image fetching.

    • Default: True

    • Example: -e IMAGE_URI_HTTPS_ONLY=False

  • IMAGE_URI_ALLOW_REDIRECTS: If set to True, image URLs are allowed to redirect.

    • Default: False

    • Example: -e IMAGE_URI_ALLOW_REDIRECTS=True

  • SUPPORTED_IMAGE_EXT: Lists the supported image file extensions.

    • Default: [".nrrd", ".nii", ".nii.gz", ".dcm"]

    • Example: -e SUPPORTED_IMAGE_EXT='[".nrrd", ".nii.gz"]'

To customize any of these environment variables, specify them when running the NIM container. Here’s an example that combines multiple environment variables:

docker run --gpus all --rm -it --ipc=host --net=host \
  -e NGC_API_KEY=$NGC_API_KEY \
  -e SUPPORTED_IMAGE_EXT='[".nrrd", ".nii.gz"]' \
  -e MAX_IMAGE_SIZE=2147483648 \
  -e IMAGE_CONNECTION_TIMEOUT=15 \
  -e IMAGE_READ_TIMEOUT=90 \
  -e DOMAIN_WHITELIST='["example.com", "trusted-domain.org"]' \
  nvcr.io/nim/nvidia/vista3d:1.0.0

This example sets custom values for supported image extensions, maximum image size, connection and read timeouts, and a domain whitelist.

FAQ (Frequently Asked Questions)#

  1. Q: What is VISTA-3D and what are its primary uses? A: VISTA-3D (Versatile Imaging SegmenTation and Annotation) is an interactive foundation model developed for precise segmentation and annotation of human anatomies in medical imaging. It’s primarily used for comprehensive body exploration, detailed sectional views, and interactive point-prompt segmentation in medical research and analysis.

  2. Q: Can VISTA-3D be used for clinical diagnosis? A: VISTA-3D is primarily a research tool. While it can assist in creating accurate ground-truth data for medical imaging analysis, it should not be used directly for clinical diagnosis without proper validation and regulatory approval.

  3. Q: What are the minimum hardware requirements to run VISTA-3D? A: VISTA-3D requires a GPU with at least 48 GB of memory. It has been validated on NVIDIA GPUs with Ampere or Hopper architecture, such as A100 and H100.

  4. Q: What types of medical images can VISTA-3D process? A: VISTA-3D is designed to work with 3D Computed Tomography (CT) images in NIfTI format.

  5. Q: How do I specify which organs or structures I want to segment? A: You can use the prompts parameter in the API payload to specify the desired classes (organs or structures) for segmentation. For example, "prompts": {"classes": ["liver", "spleen"]}.

  6. Q: Can I guide the segmentation process with user-defined points? A: Yes, VISTA-3D supports interactive point-prompt segmentation. You can provide specific points to guide the segmentation process using the points parameter in the prompts field of the API payload.

  7. Q: What output formats does VISTA-3D support? A: VISTA-3D can output segmentation masks in NIfTI (.nii.gz) and NRRD (.nrrd) formats. You can specify the desired format in the API call.

  8. Q: How do I view the segmentation results? A: You can use medical imaging software like 3D Slicer, ITK-SNAP, or MRIcroGL to view the segmentation results alongside the original CT images.

  9. Q: Is it possible to segment multiple organs simultaneously? A: Yes, you can specify multiple organs in the classes list of the prompts parameter to segment several structures in a single API call.

  10. Q: What should I do if the Docker container fails to start? A: Ensure you have the correct NVIDIA drivers installed and the NVIDIA Container Toolkit is properly set up. Also, verify that your system meets the minimum hardware requirements, especially the GPU memory.

  11. Q: What if the health check fails? A: Wait a few more seconds and try again. If the issue persists, check the docker logs using docker logs --follow vista3d for any error messages. Ensure your system resources aren’t constrained and that the container has full access to the GPU.

  12. Q: What causes an “out of memory” error during segmentation? A: Ensure your GPU has at least 48 GB of memory. If not, try using a smaller input image or segmenting fewer organs at a time. You can also try reducing the image resolution if possible.

  13. Q: How can I improve the accuracy of the segmentation? A: For challenging cases, you can use the interactive point-prompt segmentation feature. Provide strategic points on the organ boundaries to guide the model.

  14. Q: What should I do if the NGC CLI tool setup fails? A: Double-check that you’ve entered the correct API key. If issues persist, try regenerating a new API key from the NGC website. Also, ensure you have the latest version of the NGC CLI tool installed.

  15. Q: Is there a limit to the image size VISTA-3D can process? A: While there’s no strict limit, larger images require more GPU memory and processing time. If you’re having issues with very large images, consider downsampling or cropping the region of interest.

If you encounter any other issues, please refer to the NVIDIA VISTA-3D documentation or reach out to NVIDIA for support.

On this page