NVIDIA NIM currently is in limited availability, sign up here to get notified when the latest NIMs are available to download.

The Versatile Imaging SegmenTation and Annotation (VISTA) model combines semantic segmentation with interactivity, offering high accuracy and adaptability across diverse anatomical areas for medical imaging.

VISTA-3D is a domain-specialized interactive foundation model developed for segmenting and annotating human anatomies with precision and has three (3) core workflows:

  • Segment everything: Enables whole body exploration, crucial for understanding complex diseases affecting multiple organs and for holistic treatment planning.

  • Segment using class: Provides detailed sectional views based on specific classes, essential for targeted disease analysis or organ mapping, such as tumor identification in critical organs.

  • Segment point prompts: Enhances segmentation precision through user-directed, click-based selection. This interactive approach accelerates the creation of accurate ground-truth data, essential in medical imaging analysis.


Example output

VISTA-3D is powered by NVIDIA NIM, part of NVIDIA AI Enterprise. This integration offers a suite of easy-to-use microservices designed to accelerate the deployment of AI models, like VISTA-3D, across the cloud, data centers, and workstations. For more information, refer to MONAI.


A more detailed description of the model can be found in the Model Card.

Model Specific Requirements

The following are specific requirements for VISTA-3D NIM.


Please refer to NVIDIA NIM documentation for necessary hardware, operating system, and software prerequisites if you have not done so already


  • Minimum GPU memory (GB): 48 GB

  • A single NVIDIA GPU of Ampere or Hopper architecture.

  • CPU x86-64 >= 8 core (Recommended)

  • Memory >= 16 GB (Recommended)

  • Minimum Storage: 20 GB (8 GB container size)


  • Minimum NVIDIA Driver Version: 450.36.06

Quickstart Guide


This page assumes Prerequisite Software (Docker, NGC CLI, NGC registry access) is installed and set up.

  1. Pull the NIM container.

    docker pull


    The VISTA-3D model is shipped inside of the docker image, therefore no external model download is necessary.

  2. Run the NIM container.

    1docker run --rm -it --name vista3d \
    2   --runtime=nvidia -e CUDA_VISIBLE_DEVICES=0 \
    3   --shm-size=8G \
    4   -p 8008:8008 \
  3. Wait until the gRPC health check returns true before proceeding.

    curl localhost:8008/health/ready
  4. Run Inference in a new terminal.

    1curl -X POST \
    2-H "Content-Type: application/json" \
    3--output output.nrrd \
    4-d '{
    5"image": "",
    6"prompts": {
    7    "classes": ["liver", "spleen"]

Detailed Instructions

This section provides additional details outside of the scope of the Quickstart guide.

Pull Image

  1. Container image tags can be retrieved using the following command.

    ngc registry image info
     1Image Repository Information
     2    Name: medical_imaging_vista3d
     3    Display Name: VISTA-3D NIM
     4    Short Description: The VISTA-3D NIM Container accelerates ground truth data creation by streamlining segmentation and annotations with an interactive foundation model.
     5    Built By:
     6    Publisher: NVIDIA
     7    Multinode Support: False
     8    Multi-Arch Support: False
     9    Logo:
    10    Labels: NVIDIA NIM, NVIDIA AI Enterprise Supported
    11    Public: No
    12    Access Type:
    13    Associated Products: []
    14    Last Updated: Mar 15, 2024
    15    Latest Image Size: 8.77 GB
    16    Signed Tag?: False
    17    Latest Tag: 24.03
    18    Tags:
    19        24.03
  2. Pull the container image.

    docker pull
    ngc registry image pull

Launch Microservice

  1. Launch the NVIDIA VISTA-3D container in detached mode. Service will start in a few seconds.

    1docker run --rm -it --name vista3d \
    2--runtime=nvidia -e CUDA_VISIBLE_DEVICES=0 \
    3--shm-size=8G \
    4-p 8008:8008 \
  2. To observe the progress, please use the docker logs as follows.

    docker logs --follow vista3d

    The following logs indicate the Nvidia VISTA-3D server is running.

    1INFO:     Started server process [88]
    2INFO:     Waiting for application startup.
    3INFO:     Application startup complete.
    4INFO:     Uvicorn running on (Press CTRL+C to quit)

Health and Liveness checks

The container exposes health and liveness endpoints for integration into existing systems such as Kubernetes. These endpoints only return an HTTP 200 OK status code if the service is ready or live, respectively.

Run these in a new terminal.


1curl localhost:8008/health/ready


1curl localhost:8008/health/live

Run Inference

  1. Inference run on single NIfTI images. You can download any NIfTI data from or use a sample image as directly in the inference call or use MONAI core or MONAI label APIs to download data (see MONAI tutorials / APIs).

  2. Download and install 3D Slicer as the Viewer.

The model can be run using four different ways:

Run Vista-3D Model for All Labels

1curl -X POST '' \
2-H 'Content-Type: application/json' \
3--output output1.nrrd \
4-d '{
5"image": ""

You can verify the output file output1.nrrd along with the input image using 3D Slicer and all organ masks should be present.

Run Vista-3D Model for Certain Organs

1curl -X POST \
2-H "Content-Type: application/json" \
3--output output2.nrrd \
4-d '{
5"image": "",
6"prompts": {
7    "classes": ["liver", "spleen"]

You can verify the output file output2.nrrd along with the input image using 3D Slicer and mask of liver and spleen should be present.

Run Vista-3D Model for the Specific Output Format

 1curl -X POST \
 2-H "Content-Type: application/json" \
 3--output output3.nii.gz \
 4-d '{
 5"image": "",
 6"prompts": {
 7"classes": ["liver", "spleen"]
 9"output": {"extension": ".nii.gz", "dtype": "uint16"}

You can verify the output file output3.nii.gz along with the input image using 3D Slicer and mask of liver and spleen should be present.

Run the Vista Model for a Single Organ and Provide Guidance With User Points

1curl -X POST \
2-H "Content-Type: application/json" \
3--output output4.nrrd \
4-d '{
5"image": "",
6"prompts": {
7"points": {"spleen": [[52,181,126], [137,152,171]]}

You can verify the output file output4.nrrd along with the input image using 3D Slicer and the mask of the spleen should be present.

Stopping the Container

When you’re done testing the endpoint, you can bring down the container by running docker stop vista3d in a new terminal.