Advanced Usage#

Multi-GPU Support#

MAISI Nim supports multi-GPU configurations to enhance performance and speed up processing. The logic for utilizing multiple GPUs is as follows:

  • If the num_output_samples parameter is set to N and there are N visible GPUs, then N images will be processed in parallel, with each GPU handling one image.

  • If the num_output_samples parameter is larger than the number of GPUs, for example, if the number of GPUs is M and N > M, then M images will be processed in parallel. The remaining images will be queued and processed as GPUs become available.

Please note that the number of GPUs available on your system is correctly configured and visible. For example, you can set the CUDA visible devices when running a Docker container to specify which GPUs should be used:

docker run --gpus "all" -e CUDA_VISIBLE_DEVICES=0,1 \
--rm -it -e NGC_API_KEY=$NGC_API_KEY --net=host \
-e GENERATE_IMAGE_TIMEOUT=3600 nvcr.io/nim/nvidia/maisi:1.0.1

Note: If you use --gpus "device=2,3" instead of --gpus "all", be aware that the GPUs are renumbered inside the container. Specifically, GPU 2 on the host becomes GPU 0 in the container, and GPU 3 on the host becomes GPU 1 in the container. Therefore, you should still set CUDA_VISIBLE_DEVICES=0,1 to correctly map the GPUs.

Writing Data to Blob Storage#

To write generated images and masks to a blob storage, the pre_signed_url parameter should be specified. The pre_signed_url parameter is a pre-signed URL that allows you to securely upload files to a blob storage service, such as AWS S3, without requiring direct access to your storage credentials. This URL is generated with specific permissions and an expiration time, ensuring that the upload can only occur within a defined time frame and with the specified permissions.

Generate Pre-Signed URL Example#

Different cloud providers have different ways to produce pre-signed URLs. Please check their official documentation for details. Here we use AWS S3 as an example.

You can generate a pre-signed URL for an S3 bucket using the boto3 library in Python. If you are using AWS CloudShell, boto3 is pre-installed and credentials are automatically configured. For local environments, ensure that your AWS credentials are properly configured. Please refer to the official documentation for more details.

Here is an example of how to generate a pre-signed URL for uploading an object to an S3 bucket using boto3 in Python:

import boto3

# Create an S3 client
s3_client = boto3.client('s3')

# Define the bucket name, object key, and expiration time
bucket_name = '<bucket name>'

# optionally you can add a prefix to represent a folder structure like '<uploaded folder>/<uploaded filename>.zip'
object_key = '<uploaded filename>.zip'
expiration = 36000  # The URL expires in 10 hours

# Generate a pre-signed URL for uploading an object
pre_signed_url = s3_client.generate_presigned_url(
    'put_object',
    Params={'Bucket': bucket_name, 'Key': object_key},
    ExpiresIn=expiration
)

# Print the pre-signed URL
print(pre_signed_url)

Specify pre-signed URL in Payload#

After having a valid pre-signed URL, you can specify the pre_signed_url in the request payload, for example:

import requests

base_url = "http://localhost:8000"

payload = {
    "num_output_samples": 1,
    "body_region": ["abdomen"],
    "anatomy_list": ["liver", "spleen"],
    "output_size": [512, 512, 512],
    "spacing": [1.0, 1.0, 1.0],
    "image_output_ext": ".nii.gz",
    "label_output_ext": ".nii.gz",
    "pre_signed_url": "https://<example>.s3.amazonaws.com/...",
}

generation_response = requests.post(f"{base_url}/v1/maisi/run", json=payload)

if generation_response.status_code == 200:
    print("Files generated successfully and uploaded to the provided pre-signed URL!")
else:
    print("Failed to generate files. Status code: ", generation_response.status_code)
    print("Response: ", generation_response.json())

Writing Data to Local Mounted Directory#

To write generated images and masks to a local mounted directory, the local_working_dir parameter should be specified in the request payload. To enable this feature, the local directory must be mounted when running the NIM container:

docker run --gpus all --rm -it -e NGC_API_KEY=$NGC_API_KEY --net=host -v /path/on/host:/path/in/container nvcr.io/nim/nvidia/maisi:1.0.1

please ensure that the directory has the correct permissions to allow the Docker container to write to it (for example, do sudo chmod -R 777 to it). If the directory is /path/on/host/results/, then in the inference request payload, local_working_dir should be /path/in/container/results/.

Set timeout#

Set a custom timeout for image generation (default is 1800 seconds):

docker run --gpus all --rm -it -e NGC_API_KEY=$NGC_API_KEY \
-e GENERATE_IMAGE_TIMEOUT=3600 --net=host nvcr.io/nim/nvidia/maisi:1.0.1

Change Port#

The default port for the FastAPI app is 8000, and adding an environment variable NIM_HTTP_API_PORT when running the container can change it. For example:

docker run --gpus all --rm -it -e NGC_API_KEY=$NGC_API_KEY \
-e NIM_HTTP_API_PORT=12000 --net=host nvcr.io/nim/nvidia/maisi:1.0.1

Note: Some ports, such as 8001, 8002, and 8080, may already be in use by other services inside the container. Avoid setting NIM_HTTP_API_PORT to these ports to prevent conflicts.

On this page