Is this page helpful?

Legal and Acknowledgements#

This page contains the primary legal references and third-party acknowledgements for NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM).

NVIDIA AI Product Agreement#

By using this NIM, you acknowledge that you have read and agreed to the NVIDIA AI Product Agreement.

NOTICE AND DISCLAIMER: This software automatically retrieves, accesses or interacts with external materials. Those retrieved materials are not distributed with this software and are governed solely by separate terms, conditions and licenses. You are solely responsible for finding, reviewing and complying with all applicable terms, conditions, and licenses, and for verifying the security, integrity and suitability of any retrieved materials for your specific use case. This software is provided “AS IS”, without warranty of any kind. The author makes no representations or warranties regarding any retrieved materials, and assumes no liability for any losses, damages, liabilities or legal consequences from your use or inability to use this software or any retrieved materials. Use this software and the retrieved materials at your own risk.

Open Source Software License Acknowledgements#

NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM) (NIM LLM and VLM) is built on the work of open source communities whose ongoing innovation drives the LLM inference ecosystem forward. NIM LLM and VLM runs on one of two open source inference engines, vLLM or SGLang, selected by the container image you deploy. NVIDIA is grateful for the collaboration, contributions, and shared commitment to advancing AI infrastructure that these projects represent.

vLLM#

vLLM is the inference engine packaged in the vLLM-based NIM LLM and VLM container images. The project provides a high-throughput, memory-efficient serving engine for large language models, and those images package vLLM directly as their inference backend.

NVIDIA recognizes the vLLM community for:

Foundational inference technology. vLLM introduced PagedAttention, a novel memory management algorithm for KV cache, and has subsequently implemented cutting-edge theory such as continuous batching, automatic prefix caching, and chunked prefill into a production-grade, open source engine. These capabilities form the backbone of every NIM LLM and VLM deployment that runs on the vLLM backend.
Sustained collaboration. NVIDIA engineers contribute upstream to vLLM, and NVIDIA is dedicated to continuing this investment. Improvements flow in both directions - NVIDIA contributes optimizations to the vLLM project and community-driven innovations are available to NIM LLM and VLM users.
An open development model. The vLLM project maintains a transparent development process that welcomes contributors across organizations. This openness accelerates the pace of innovation for the entire ecosystem.

For vLLM documentation, refer to the vLLM project documentation.

For details on how NIM LLM and VLM integrates with its inference backend, refer to Architecture.

SGLang#

SGLang is the inference engine packaged in SGLang-based NIM LLM and VLM container images. NVIDIA recognizes the SGLang community for its open source serving engine and collaboration on LLM and VLM inference. For backend features and configuration, refer to the SGLang project documentation.

Open Source Projects#

NIM LLM and VLM container images include other open source software components besides the inference engines. Each published container image has its own open source archive page, which lists the open source packages in that image and links to their source code:

https://opensource.nvidia.com/oss/teams/nim/<repository>/<tag>:linux-<arch>/index.html

Here <repository> and <tag> are the NGC repository name and tag of the image, and <arch> is amd64 or arm64. For example, the page for nvcr.io/nim/nvidia/vllm-model-free-nim:<tag> on an x86-64 host is https://opensource.nvidia.com/oss/teams/nim/vllm-model-free-nim/<tag>:linux-amd64/index.html. The /opt/nim/NOTICE file in each NIM container image points to the open source archive.

The following components are built from source or modified by NVIDIA, or have license terms that call for a specific notice. Their source code is available from the archive page of each image that contains them or, where noted, inside the image.

Component	In images	License	Notes
nginx	vLLM-based and SGLang-based	BSD-2-Clause	Built from the nginx.org source release. Runs as the HTTP proxy in front of the inference server.
FFmpeg libraries	vLLM-based and SGLang-based	LGPL-2.1-or-later	Built by NVIDIA from the FFmpeg source release, with backported upstream security fixes that are archived with the source, as shared libraries in `/usr/local/lib` whose names end in `-nimlgpl`. They are configured without GPL or non-free components and without network support, and they decode audio only, from a fixed list of audio decoders. The configure line is embedded in the libraries.
PyAV	vLLM-based and SGLang-based	BSD-3-Clause	Built from the unmodified PyPI source release and linked dynamically against the FFmpeg libraries above.
PyNvVideoCodec	vLLM-based	MIT; the FFmpeg libraries it bundles are LGPL-2.1-or-later	Video decoding for multimodal input; decoding runs on the GPU. NVIDIA builds it with FFmpeg libraries configured without any codecs.
Mooncake transfer engine	vLLM-based and SGLang-based	Apache-2.0; the libraries it bundles are under the BSL-1.0, BSD-3-Clause and MIT licenses	Rebuilt by NVIDIA from the upstream release with newer Go toolchain and Go module versions. Its etcd client library statically links the Go standard library (BSD-3-Clause) and Go modules under the Apache-2.0, BSD-3-Clause and MIT licenses.
CRIU	vLLM-based, x86-64 only	GPL-2.0-only; LGPL-2.1 for its `lib/` directory	Installed by the upstream vLLM container image for vLLM’s optional snapshot feature, which NIM LLM and VLM does not use. The image includes its source code at `/usr/share/doc/criu/source/criu.tar.gz`.
tiktoken encodings (`cl100k_base`, `o200k_base`)	vLLM-based and SGLang-based	Published by OpenAI for its MIT-licensed tiktoken library	Unmodified tokenizer data in `/opt/nim/tiktoken_encodings/`, so that models that need these encodings run without network access.

For questions about the open source software in NIM LLM and VLM, contact oss-requests@nvidia.com.