> This page is for MAISI, version 1.0.0.
> For other versions, use one of these documentation indexes:
> - 1.0.1 (Latest) (default): https://docs.nvidia.com/nim/medical/maisi/1.0.1/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/medical/maisi/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/medical/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/medical/_mcp/server.

# Model fine-tuning

When MAISI NIM is running, by default, it automatically downloads the MAISI model bundle and weights from NGC into the NIM container. However, if you wish to fine-tune MAISI and continue using it within NIM, here is detailed guide to utilize a custom model bundle and weights in MAISI NIM.

## Fine-tune the Model

Please follow [this tutorial](https://github.com/Project-MONAI/tutorials/blob/main/generation/maisi/maisi_diff_unet_training_tutorial.ipynb) on Project MONAI to learn about how to train a MAISI model.

## Use the fine-tuned model in NIM

Once you have the fine-tuned MAISI bundle at your local storage, you simply need to mount it to `/opt/bundle` and set the correct model manifest profile for it by setting `NIM_MANIFEST_PROFILE` environment variable.

```bash
-e NIM_MANIFEST_PROFILE=43d0baebee73b7bdc2f9c7edf5c726b2323b4f7283eab92fab893d451592656d \
-v /path/to/fine-tuned/bundle:/opt/bundle \
```

Here is the full command to run the NIM with fine-tuned model.

```bash
docker run --rm -it --name maisi \
   --runtime=nvidia -e CUDA_VISIBLE_DEVICES=0 \
   --shm-size=8G \
   -p 8000:8000 \
   -e NGC_API_KEY=$NGC_API_KEY \
   -e NIM_MANIFEST_PROFILE=43d0baebee73b7bdc2f9c7edf5c726b2323b4f7283eab92fab893d451592656d \
   -v /path/to/local/bundle:/opt/bundle \
   nvcr.io/nim/nvidia/maisi:1.0.0
```


Here is the list of available profiles:

* `56df3d486bbd1c8ac4a1de4d613d4037927bd462d751bf4fe9cbeb8d14890cb8`: (default profile) downloads and use MAISI bundle on NGC.
* `43d0baebee73b7bdc2f9c7edf5c726b2323b4f7283eab92fab893d451592656d`: utilizes the user provided MAISI bundle (fine-tuned model) at `/opt/bundle`.

## Troubleshooting

If you encounter issues while using the fine-tuned model, here are some common troubleshooting steps:

### Common Issues

1. Model not loading: Ensure that the path to the fine-tuned bundle is correct and that the bundle is properly mounted to /opt/bundle. You may see this error if the bundle is not mounted under `/opt/bundle`:

   ```bash
   nimlib.exceptions.ManifestDownloadError: Error downloading manifest to cache: /opt/nim/.cache error: Invalid Repo ID: path does not exist: /opt/bundle; repo_id: RepoId { repo_path: "/opt/bundle", revision: None, protocol: Some("local") }
   ```

   and this one if the local directory (which is mounted to `/opt/bundle`) does not contain the necessary bundle files:

   ```bash
   nimlib.exceptions.ManifestDownloadError: Error downloading manifest to cache: /opt/nim/.cache error: Object not found
   ```

2. Incorrect Profile: Verify that the NIM_MANIFEST_PROFILE environment variable is set to the correct profile ID. The profile ID needs to be one of the above-mentioned IDs. Otherwise, you may end up with some automatic profile selection. For fine-tuned model, you can check in the logs to see if the profile is properly set:

   ```bash
   "Matched profile_id in manifest from env NIM_MANIFEST_PROFILE to: 43d0baebee73b7bdc2f9c7edf5c726b2323b4f7283eab92fab893d451592656d"
   ```

3. Docker Permissions: Make sure you have the necessary permissions to run Docker commands and access the specified directories.

### Logs and Debugging

To view logs and debug issues, you can use the following Docker commands:

View Logs:

```bash
docker logs maisi
```

Access Container Shell:

```bash
docker exec -it maisi /bin/bash
```