Is this page helpful?

NIM LLM 1.x to NIM LLM and VLM 2.0 Migration Guide#

NIM LLM and VLM 2.0 uses a new architecture in which each container image bundles one open-source inference engine, vLLM or SGLang, in place of the 1.x multi-backend design. As a result, it differs significantly from NIM LLM 1.x. This guide highlights the main areas that require changes during migration and links to the relevant 2.0 documentation for full details.

Migration Checklist#

Use the following checklist to review the main migration tasks.

General#

Start with these high-level changes:

  • Update the image tag to 2.1.4.

  • Review the new architecture, including which inference engine (vLLM or SGLang) your image bundles.

CLI#

Review the CLI changes that affect deployment and startup behavior:

  • There are minor changes and additions to the CLI. Refer to CLI Reference for more details on the new CLI.

  • Notably, the nim-run CLI command is replaced with the new nim-serve command.

Configuration#

Review configuration changes before you update deployment manifests or startup commands:

  • Migrate to the new configuration system. Refer to Advanced Configuration for details.

  • Audit environment variables against the 2.0 environment variable reference. Many 1.x variables have been removed. Use the canonical NIM_* variables where they exist. For everything else, pass the bundled engine’s native arguments (vLLM or SGLang, depending on the image) through nim-serve. Backend-specific flags must match the engine in your image.

  • Configure the inference engine by passing its CLI arguments directly to nim-serve. Refer to the nim-serve section of CLI Reference for an example. For a full description of the available configuration options, refer to the vLLM engine arguments or the SGLang server arguments, depending on your image.

API and Inference Requests#

Update client integrations to match the new request and response structure:

  • Review the new API Reference and migrate clients to the OpenAI-compatible request and response structure exposed by the bundled engine.

  • Notably, the nvext extension object in the request body is removed. Move sampling parameters from nvext to top-level request fields.

  • For extension sampling fields beyond the OpenAI schema, refer to the vLLM sampling parameters or SGLang sampling parameters. Both backends accept min_tokens in OpenAI-compatible Chat Completions and Completions requests. SGLang maps this field to min_new_tokens internally.

Tool Calling#

Review tool calling changes if your deployment uses function or tool invocation:

Models and Profiles#

Review model selection and profile behavior before you migrate existing deployments:

  • Replace the NIM_MODEL_NAME environment variable with NIM_MODEL_PATH to configure the model to download and serve. Refer to Model Download and Model-Free NIM for additional details.

  • Automatic profile selection now uses model memory estimation heuristics to choose a suitable profile. Refer to Model Profiles and Selection for details.

LoRA Support#

Review these changes if you deploy LoRA adapters:

  • Convert LoRA adapters to Hugging Face PEFT format. NeMo-format adapters are no longer supported.

  • Refer to Fine-Tuning with LoRA for details on configuring NIM for LoRA.

Observability and Logging#

Update logging and metrics collection to match the 2.0 runtime:

  • Update dashboards and alerts to use the bundled engine’s native Prometheus metrics: vllm:* for vLLM or sglang:* for SGLang. Metric names differ between engines. Refer to Logging and Observability.

  • Use NIM_LOG_LEVEL and NIM_JSONL_LOGGING for logging configuration.

  • Refer to Logging and Observability — Metrics for more details.