NIM Operator Quick Start

Follow the examples in the tiles to deploy NIM. NVIDIA highly recommends that you cache models for low inference latency and faster auto-scaling.

Non-LLM NIM

Domain-specific, RAG, BioNeMo, and Riva.

Docs | NIMCache | NIMService

LLM-specific NIM

Optimized NVIDIA NIM.

Docs | NIMCache | NIMService

Multi-LLM compatible NIM deployment

Broad range of LLMs, such as from Hugging Face.

Docs | NIMCache | NIMService

Multi-node NIM deployment

Deploy large LLMs across several interconnected nodes.

Docs | NIMCache | NIMService

NIM on KServe serverless deployment

Serverless deployment, scale-to-zero, and canary rollouts.

Docs | NIMCache | NIMService

NIM with DRA

Advanced GPU allocation and scheduling.

Docs | NIMCache | NIMService

NIM with LoRA

Enhance LLMs with domain-specific adapters.

Docs | NIMCache | NIMService