> This page is for version 1.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/alpamayo/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/alpamayo/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/alpamayo/_mcp/server.

# Support Matrix

Alpamayo1.5 NIM currently supports one GPU per container.

## Validated GPUs

The following GPUs have profile-specific validation and published K caps.

| GPU                      | GPU memory | Validated precisions   | K configuration                     |
| ------------------------ | ---------- | ---------------------- | ----------------------------------- |
| H100 80 GB (SXM or PCIe) | 80 GB      | `bf16`, `fp8`, `w4a16` | Runtime-selected; maximum K is `15` |
| H100 NVL (94 GB)         | 94 GB      | `bf16`, `fp8`, `w4a16` | Runtime-selected; maximum K is `15` |
| H200                     | 141 GB     | `bf16`, `fp8`, `w4a16` | Runtime-selected; maximum K is `20` |
| RTX 4090                 | 24 GB      | `fp8`, `w4a16`         | Runtime-selected; maximum K is `1`  |
| RTX 3090                 | 24 GB      | `w4a16`                | Runtime-selected; maximum K is `1`  |
| RTX 3090 Ti              | 24 GB      | `w4a16`                | Runtime-selected; maximum K is `1`  |

A100 is not profile-specifically validated for this release. It follows the compatible-unvalidated path in the following section: begin with a generic BF16 or W4A16 profile at `K=1` and confirm startup and inference on the exact deployment GPU.

## Profiles

| Profile | Precision              | K values                               | Validated GPUs                                              |
| ------- | ---------------------- | -------------------------------------- | ----------------------------------------------------------- |
| `bf16`  | BF16                   | Default `K=1`; explicit runtime K list | H100 80 GB, H100 NVL, H200                                  |
| `fp8`   | ModelOpt FP8 VLM       | Default `K=1`; explicit runtime K list | H100 80 GB, H100 NVL, H200, RTX 4090                        |
| `w4a16` | ModelOpt W4A16/AWQ VLM | Default `K=1`; explicit runtime K list | H100 80 GB, H100 NVL, H200, RTX 4090, RTX 3090, RTX 3090 Ti |

## Compatible Unvalidated GPUs

The released precision profiles are generic rather than compiled for one GPU name. A GPU outside the validated table can start when NVML reports a compatible CUDA compute capability and both its total and currently free memory meet the profile minimum:

| Profile | Minimum compute capability | Minimum GPU memory |
| ------- | -------------------------- | ------------------ |
| `bf16`  | 8.0                        | 30 GB              |
| `fp8`   | 8.9                        | 20 GB              |
| `w4a16` | 8.0                        | 20 GB              |

This compatibility path includes GPUs such as RTX 6000 Ada when they satisfy the table, but it does not imply profile-specific performance validation. Exact GPU-family or SM requirements remain enforced for any internal profile that contains hardware-specific artifacts. Pre-Ampere GPUs with compute capability below 8.0 are not compatible with these profiles.

## Profile Selection

The NIM automatically selects a compatible profile for the visible GPU. Use `NIM_PRECISION` to request a precision:

```bash
-e NIM_PRECISION=fp8
```

If a requested precision is not compatible with the visible GPU, startup fails with a profile-selection error. The manifest exposes generic precision profiles; startup checks map those profiles onto the detected GPU class, compute capability and memory. Auto selection prefers BF16 when at least 30 GB is available, then chooses a compatible quantized profile for smaller GPUs.

The active trajectory count defaults to `K=1`. Set `NIM_ALPAMAYO_TRAJ_SAMPLES` to the exact K values that this container should warm and serve:

```bash
-e NIM_ALPAMAYO_TRAJ_SAMPLES=1,10
```

Use a single value, such as `NIM_ALPAMAYO_TRAJ_SAMPLES=10`, for a fixed high-K deployment, or an explicit list such as `1,2,3` when clients may request multiple K values during serving. Each configured value is warmed during startup and retained for CUDA graph performance, so avoid configuring values that clients do not request. Values above a validated GPU's published cap fail during profile selection before server startup. Unlisted GPUs default to `K=1` and may explicitly request values up to the global experimental limit of 32; successful startup is not a guarantee that every larger K value fits that GPU.

Requests outside the selected K list return HTTP 400 with `type="invalid_request_error"`. For example, `num_traj_samples=2` on a container configured with `NIM_ALPAMAYO_TRAJ_SAMPLES=1,10` returns an error that the profile supports only those values.

## Unlisted GPU and Out-of-Memory Troubleshooting

`K` is the number of trajectory candidates returned by one request. Every value listed in `NIM_ALPAMAYO_TRAJ_SAMPLES` is warmed during startup and retains shape-specific CUDA graph/cache state. There is no portable fixed memory cost per K because it depends on the precision, GPU architecture, prompt shape, and configured K list.

For an unlisted GPU, begin with one quantized K=1 profile:

```bash
-e NIM_PRECISION=w4a16 \
-e NIM_ALPAMAYO_TRAJ_SAMPLES=1
```

Use `fp8` instead of `w4a16` when the GPU has compute capability 8.9 or newer. After the container reaches readiness and completes an untimed request, increase K one value at a time. A list such as `1,2,4` consumes more retained state than a single fixed value such as `4`.

If startup or inference reports CUDA out-of-memory:

1. Stop sending requests and check `/v1/health/ready`.
2. Restart with `NIM_ALPAMAYO_TRAJ_SAMPLES=1`.
3. Select `w4a16` or, on compute capability 8.9 or newer, `fp8`.
4. Remove unused K values and stop other processes consuming GPU memory.
5. Increase K gradually only after the smaller configuration is stable.

## Latency Notes

For customer trajectory serving, prefer direct gRPC on port `50051` when the client can use the binary API. This avoids the HTTP base64 and JSON adapter path and reduces serving overhead. Send JPEG images when possible to use the fastest image decode path.

Multi-GPU deployment is not currently used for Alpamayo1.5 latency profiles. The recommended deployment shape is one NIM container per GPU.