Is this page helpful?

Release Notes#

Release 2.1.0#

This release contains model updates outlined in the following sections.

vLLM Model-Free NIM#

This release introduces the vLLM model-free NIM, which allows you to run any model supported by vLLM without a model-specific container image. For more information, refer to Model-Free NIM.

For GPU support, refer to the support matrix for Supported Model-Free NIM Images.

Note the following vLLM model-free NIM limitations:

  • LoRA is not supported.

  • This release uses the vLLM 0.24.0 backend.

  • Reasoning and tool-call parsers are disabled by default.

  • Backend-native parameters that you pass through command-line arguments, the NIM_PASSTHROUGH_ARGS environment variable, or tuning configurations must use the vLLM parameter names.

  • The runtime manifest engine tag is vllm instead of sglang.

  • Dynamically generated profile IDs and checksums can change. Do not reuse existing SGLang profile IDs with the vLLM model-free NIM.

SGLang Model-Free NIM#

This release introduces the SGLang model-free NIM, which allows you to run any model supported by SGLang without a model-specific container image. For more information, refer to Model-Free NIM.

For GPU support, refer to the support matrix for Supported Model-Free NIM Images.

Note the following SGLang Model-Free NIM limitations:

  • Video inference requires an external FFmpeg/libav mount. FFmpeg 8 is preferred. Set the following environment variables to mount FFmpeg/libav:

    -e PATH=/opt/external-ffmpeg/bin:$PATH
-e LD_LIBRARY_PATH=/opt/external-ffmpeg/lib:$LD_LIBRARY_PATH
-v /path/to/external-ffmpeg:/opt/external-ffmpeg:ro

  • To enable tool-call and reasoning, add the following passthrough args with the desired parsers:

    -e NIM_PASSTHROUGH_ARGS="--tool-call-parser qwen3_coder --reasoning-parser qwen3"

  • The environment variable NIM_SERVED_MODEL_NAME renames the base model only. The name is not used as a prefix. LoRA adapter names remain as the adapter directory names under NIM_PEFT_SOURCE.

  • The environment variables NIM_HEALTH_PORT and NIM_MAX_MODEL_LEN are not supported by this container.

For information about past updates and older versions, refer to the previous release notes.