# Model Profiles and Selection[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#model-profiles-and-selection 'Link to this heading')

A model profile is a pre-validated inference configuration that NIM LLM and VLM selects at container startup.

Every model-specific NIM container ships with a **model manifest** — a catalog of one or more **profiles**. Each profile is defined by tags for the backend engine, model precision, tensor parallelism size (TP), pipeline parallelism size (PP), and LoRA support.

For model-free NIM, the model manifest is generated at runtime with generic profiles that can deploy the NIM across a wide range of system configurations.

At container startup, NIM selects exactly one profile from the manifest. The selected profile determines which model files are downloaded and how the inference backend is launched.

## Profile Naming Convention[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#profile-naming-convention 'Link to this heading')

Profile descriptions are generated from the tags in the container’s model manifest. A common base form is:

```
<backend>-<precision>-tp<N>-pp1[-lora]
```

Where:

- `<backend>` is the inference engine (`vllm`, `sglang`, or `trtllm`). The backend must match the inference engine in the container image you run. For example, the SGLang container image only loads `sglang-*` profiles.

- `<precision>` is the quantization format (`bf16`, `fp8`, `mxfp4`, or `nvfp4`)

- `tp<N>` is the tensor parallelism degree (number of GPUs)

- `pp1` indicates single-stage pipeline parallelism

- The `-lora` suffix indicates the profile supports LoRA adapter loading

For example, `vllm-bf16-tp4-pp1-lora` uses BF16 precision across four GPUs with LoRA support on the vLLM backend, and `sglang-bf16-tp4-pp1` runs the equivalent configuration on the SGLang backend.

Descriptions can include additional tag values, such as a GPU name, a performance variant (`throughput` or `latency`), or a minimum-memory value. The base form above is not a template for reconstructing an exact selector.

Copy the complete description printed by `list-model-profiles` for the image version you plan to deploy, or use the full profile ID. The optimized-profile tables in the [Support Matrix for NIMs](https://docs.nvidia.com/nim/large-language-models/latest/reference/support-matrix.html#support-matrix) provide full IDs to distinguish profiles that share the same precision and parallelism.

## List Available Profiles[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#list-available-profiles 'Link to this heading')

To list the profiles that a container can run, complete the following steps:

Run the `list-model-profiles` command:

```
docker run --rm --gpus=all \
  <nim_llm_image> \
  list-model-profiles
```

Example output:

```
MODEL PROFILES
- Compatible with system and runnable:
  - dcec66a50892315842bdc46d5b2d8648fed3fe3d3382437f0a811c56eff8c39c (vllm-bf16-tp1-pp1) [requires >=18 GB/gpu]
  - With LoRA support:
    - d66193b819d2bc2ae40aefcec0da5997b5f9187dd79b8155ec111b16999d18e0 (vllm-bf16-tp1-pp1-feat_lora) [requires >=22 GB/gpu]
- Compatible with system but low memory:
  - a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2 (vllm-bf16-tp1-pp1) [requires >=45 GB/gpu, try --max-model-len=4096 to reduce to >=30 GB/gpu]
- Incompatible with system:
    - 27af459c9caa0f9b34d5e07e5962960df6b0120df2039d06148e0e63595195e5 (vllm-bf16-tp2-pp1)
    - 30d16624c8100d40e6cde3af7f4e4ff6028f776e92efdcf09fcb515ae65662c0 (vllm-bf16-tp4-pp1)
    - 6f888502f35dc189f8c67f3e11174028a4ce42e92868e6a0ca10ef1d84953874 (vllm-bf16-tp8-pp1)
```

Note

The example above shows output from a vLLM container image. The SGLang container image emits profiles with `sglang-*` descriptions instead (for example, `sglang-bf16-tp1-pp1`).

Each profile has:

- **Profile ID**: A unique 64-character profile ID.

- **Profile description**: A human-readable string constructed by joining tag values with hyphens (for example, `vllm-fp16-tp1-pp1`).

- **Memory annotation**: An estimated VRAM requirement per GPU, shown in brackets (for example, `[requires >=18 GB/gpu]`).

## Memory-Based Profile Classification[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#memory-based-profile-classification 'Link to this heading')

NIM estimates the GPU VRAM required by each profile and classifies it into one of three categories based on the available memory on the system:

| Category | Meaning | Action |
| --- | --- | --- |
| **Compatible** | Estimated VRAM fits within available GPU memory | Profile can be selected and deployed |
| **Low memory** | Model weights fit, but full context length exceeds available memory | Profile can run with a reduced `--max-model-len`. The listing includes a suggested value. |
| **Incompatible** | Model weights alone exceed available GPU memory | Profile cannot run on this hardware. Consider using a profile with higher tensor parallelism or a quantized precision. |

If a profile is classified as **low memory**, the listing output includes a suggestion. For example:

```
[requires >=45 GB/gpu, try --max-model-len=4096 to reduce to >=30 GB/gpu]
```

To apply the suggestion, pass the `--max-model-len` argument:

```
docker run --rm -it --gpus=all \
  -p 8000:8000 \
  <nim_llm_image> \
  --max-model-len 4096
```

Note

Reducing `--max-model-len` limits the maximum sequence length (input + output tokens) per request. Choose a value that fits your use case.

## How Profile Selection Works[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#how-profile-selection-works 'Link to this heading')

NIM uses a priority-ordered selection chain to decide which profile to use. The chain is evaluated top-to-bottom; the first selector that produces a match wins.

| Priority | Selector | Trigger | Description |
| --- | --- | --- | --- |
| 1 (highest) | Default profile selector | `NIM_MODEL_PROFILE="default"` | Selects the first hardware-compatible profile using backend priority. |
| 2 | Environment-based profile selector | `NIM_MODEL_PROFILE=<id-or-name>` | Matches an explicit profile by checksum or description. |
| 3 | Memory-aware profile selector | _(automatic)_ | Estimates VRAM requirements for each profile and filters out profiles that exceed available GPU memory. Prefers non-LoRA profiles unless LoRA is enabled. |
| 4 (lowest) | Manifest profile selector | _(no env var set)_ | Uses `profile_selection_criteria` from the manifest to pick a profile automatically based on hardware compatibility (GPU device, VRAM) and parallelism settings. |

The memory-aware selector runs automatically as part of the selection chain. It estimates GPU memory requirements for each candidate profile by analyzing model weights, KV cache, activations, and overhead. Profiles that do not fit in available GPU memory are excluded from selection.

## Select a Profile[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#select-a-profile 'Link to this heading')

The method you use to select a profile depends on your requirements and environment. You can allow NIM to pick a compatible profile automatically, or you can specify the exact profile by ID or by description.

### Automatic Selection (Default)[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#automatic-selection-default 'Link to this heading')

If you do not set `NIM_MODEL_PROFILE`, NIM automatically selects the best compatible profile from the manifest based on your hardware (GPU device, available VRAM, estimated memory requirements, and parallelism constraints).

To start the container with automatic profile selection, run the container without `NIM_MODEL_PROFILE`:

```
docker run --rm -it --gpus=all \
  -p 8000:8000 \
  <nim_llm_image>
```

### Intelligent Default Selection[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#intelligent-default-selection 'Link to this heading')

Setting `NIM_MODEL_PROFILE` to `"default"` triggers intelligent default selection. NIM picks the best compatible profile based on the following criteria:

1. Hardware compatibility (GPU device, VRAM)

2. Backend priority

3. LoRA configuration

To use intelligent default selection, set `NIM_MODEL_PROFILE` to `"default"` and start the container:

```
docker run --rm -it --gpus=all \
  -e NIM_MODEL_PROFILE="default" \
  -p 8000:8000 \
  <nim_llm_image>
```

### Explicit Selection by Profile ID[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#explicit-selection-by-profile-id 'Link to this heading')

Specify the full 64-character profile ID from the manifest of the image you plan to deploy. This selects that exact profile without reconstructing its description. Check the ID again when changing image versions: a profile change can produce a different ID.

To select a profile by ID, set `NIM_MODEL_PROFILE` to the 64-character profile ID:

```
docker run --rm -it --gpus=all \
  -e NIM_MODEL_PROFILE="70edb8bb9f8511ce2ea195e3caebcc3c7191dc27fea0c8d4acf9c0d9a69e43cd" \
  -p 8000:8000 \
  <nim_llm_image>
```

### Explicit Selection by Profile Description (Friendly Name)[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#explicit-selection-by-profile-description-friendly-name 'Link to this heading')

If the value of `NIM_MODEL_PROFILE` is not a valid Profile ID, NIM tries to match it against the profile description — a human-readable string constructed from ordered profile tags.

To select a profile by description, set `NIM_MODEL_PROFILE` to the profile description:

```
docker run --rm -it --gpus=all \
  -e NIM_MODEL_PROFILE=vllm-fp16-tp1-pp1 \
  -p 8000:8000 \
  <nim_llm_image>
```

Note

On an SGLang container image, use the matching `sglang-` profile description (or its hash) instead, for example `sglang-fp16-tp1-pp1`.

Tip

Use `list-model-profiles` to discover the exact profile IDs and descriptions available in your container.

## Configuration Precedence[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#configuration-precedence 'Link to this heading')

`NIM_MODEL_PROFILE` selects a model profile. The following sources override its settings, from highest to lowest priority: direct container arguments, `NIM_PASSTHROUGH_ARGS`, NIM environment variables that map to backend arguments, and `runtime_config.json`. For the complete order, refer to [Advanced Configuration](https://docs.nvidia.com/nim/large-language-models/latest/reference/advanced-configuration.html#advanced-configuration).

For example, if a profile specifies `tp=2` but the user also passes the backend’s tensor-parallelism flag set to 4 (such as the vLLM `--tensor-parallel-size 4` argument), the backend launches with TP=4.

Important

When backend arguments override profile settings, the overridden values are resolved **before** model download. NIM selects and downloads the profile that matches the final resolved configuration, so the downloaded model files always match the launch configuration.

## Pass Backend CLI Arguments[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#pass-backend-cli-arguments 'Link to this heading')

You can also control parallelism and other settings by passing backend CLI arguments directly to the container. This works with both vLLM and SGLang, but the argument names are backend-specific.

To pass a vLLM tensor-parallelism argument, append the backend flag to the container command. The following example uses vLLM arguments:

```
docker run --rm -it --gpus=all \
  -p 8000:8000 \
  <nim_llm_image> \
  --tensor-parallel-size 2
```

Common vLLM CLI arguments include the following:

| vLLM CLI Argument | Purpose | Default |
| --- | --- | --- |
| `--tensor-parallel-size` | Number of tensor-parallel GPUs | 1 |
| `--pipeline-parallel-size` | Number of pipeline-parallel stages | 1 |
| `--enable-lora` | Enable LoRA adapter support | Disabled |

Note

On an SGLang image, the equivalent flag names differ. Refer to the SGLang documentation for the corresponding backend arguments. For a Kubernetes workload, pass the arguments through the container `args` field. For a NIM Operator deployment that does not set `spec.multiNode`, use `spec.args`. Do not use `spec.args` for a multi-node NIMService; the current LeaderWorkerSet path does not propagate it to the leader or worker NIM containers. The direct-argument interfaces preserve any `NIM_PASSTHROUGH_ARGS` value provided by the image. For details, refer to [Advanced Configuration](https://docs.nvidia.com/nim/large-language-models/latest/reference/advanced-configuration.html#advanced-configuration).

## Changes from NIM LLM 1.x[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#changes-from-nim-llm-1-x 'Link to this heading')

The following profile selection mechanisms from NIM LLM version 1.x are **no longer supported**:

| Removed Feature | 1.x Example |
| --- | --- |
| Custom profile selectors | `-e NIM_CUSTOM_SELECTOR_CLASSES="my_selector.MyCustomSelector"` |
| LLM-based profile selector (backend priority chain) | Automatic backend priority: TensorRT-LLM > vLLM > SGLang |
| Tag-based profile selector | `-e NIM_TAGS_SELECTOR="llm_engine=vllm,tp=1"` |

Tip

Use `NIM_MODEL_PROFILE` with a profile ID or description as a replacement for these deprecated mechanisms. For further guidance, refer to the [1.x Migration Guide](https://docs.nvidia.com/nim/large-language-models/latest/reference/1.x-migration-guide.html.md#x-migration-guide).

## Next Steps[#](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-profiles-and-selection.html.md#next-steps 'Link to this heading')

After you select a profile, download model artifacts and start serving:

- [Model Download](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-download.html.md#model-download) — Details on NIM model download functionality

- [Environment Variables](https://docs.nvidia.com/nim/large-language-models/latest/reference/environment-variables.html#environment-variables) — Full reference for `NIM_MODEL_PROFILE` and other configuration variables

- [About Model-Free NIM](https://docs.nvidia.com/nim/large-language-models/latest/deployment/model-free-nim.html.md#model-free-nim) — Runtime manifests and generic profiles for model-free containers