NVIDIA NIM for LLM and VLM Documentation#
NVIDIA NIM for Large Language Models (LLM) and Vision Language Models (VLM) (NIM LLM and VLM) is a production-ready way to run large language models and vision language models with NVIDIA inference microservices (NIMs). NIM LLM and VLM brings state-of-the-art LLM and VLM serving to enterprise and developer workflows with validated containers, curated weights, and direct alignment with upstream inference engines. It is designed for teams that do not have the bandwidth to track the fast-moving model-serving ecosystem. Depending on the offering, it can prioritize rapid access to newly released models, validated best-in-class performance, or enterprise lifecycle guarantees for long-term production deployments.
NIM LLM and VLM is built on vLLM, an open source, high-throughput inference engine that NVIDIA actively contributes optimizations to upstream. It uses that engine directly rather than layering heavy abstractions over it, so engine optimizations, new CUDA versions, and hardware support arrive in weeks rather than months. For a list of open source acknowledgements, refer to Open Source Acknowledgements.
Skip Ahead#
Use the following resources to begin a deployment or check what the current release supports:
Set up your environment, pull the container, and run your first model.
Supported models, profiles, and hardware platforms for the latest release.
Diagnose common deployment issues, including GPU out-of-memory errors and CUDA driver failures.
New features, bug fixes, and known issues for the current release.
Use Cases#
NIM LLM and VLM enables organizations to reliably deploy generative AI capabilities across a wide range of applications at scale, including the following use cases:
Deliver human-like language understanding and responsiveness in conversational applications.
Produce high-quality content or distill lengthy articles into concise summaries.
Understand user sentiments in real time to drive better business decisions.
Remove language barriers with efficient and accurate translation services.
Build applications that reason over images, audio, video, and text with supported multimodal models.
The potential applications of NIM LLM and VLM are vast, spanning across many industries.
Core Concepts#
Explore the core concepts to understand NIM LLM and VLM:
How the NIM and NIM Certified offerings differ in publication speed, hardware validation, and enterprise lifecycle guarantees.
The packaging that long-lived production deployments require, including curated model weights, continuous CVE patching, OSRB compliance, FedRAMP-ready production branches, and built-in health and observability APIs.
Containers that ship a model-specific manifest, curated weights, validated quantization profiles, and tuned runtime configurations. Use these for standard, widely-used models such as Llama 3.
Containers that serve a model you configure at runtime from NGC, Hugging Face, Amazon S3, Google Cloud Storage, or a local directory. Use these for custom-trained models, day-zero model support, or to reduce container approvals to a single image.
Core Components#
The NIM LLM and VLM container is designed for deployment simplicity, startup reliability, and inference performance parity with upstream engines. The architecture is organized into the following three primary components:
The entry point that orchestrates startup, applies configuration from CLI flags, environment variables, and runtime configs, and injects enterprise features such as custom middleware and Low-Rank Adaptation (LoRA) adapters. It also starts the thin proxy for health checks, inference routing, and TLS termination.
Handles model licensing, hardware-aware profile selection, model downloading, and NIM-specific management API endpoints, for example, health and readiness checks.
The core engine that executes model inference and provides native OpenAI-compatible API endpoints.
Unlike the 1.x architecture that bundled multiple backends into a single container, NIM LLM and VLM version 2.0 embraces a one container, one backend philosophy for predictable behavior and direct access to upstream features. If you are upgrading from 1.x, refer to the 1.x to 2.0 Migration Guide.