Getting Started

Quickstart

Get a Dynamo OpenAI-compatible endpoint running locally on an NVIDIA GPU or Intel XPU.

View as Markdown

Choose Your Path

Quickstart

You’re here. Container fast path.

Local Installation

Full walkthrough — PyPI, configuration.

Kubernetes

Kubernetes-native production path.

Build from Source

For contributors against main.

Dynamo is backend-agnostic and Kubernetes-native without being Kubernetes-only. Use this container path to try the same frontend/router/worker stack locally; use the Kubernetes path when you want the operator, CRDs, Gateway API integration, autoscaling, scheduling, and cluster lifecycle management.

Run Dynamo Locally

1

Choose and install a build

Containers have all dependencies pre-installed. Pick your backend:

$docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0

Hugging Face token required for gated models. Llama, Kimi, Qwen-VL, and other gated models require HF_TOKEN in your environment and accepting the model card’s license on huggingface.co. Set export HF_TOKEN=hf_… before launching.

The remaining steps run inside the selected container. If you installed an NVIDIA wheel instead, run the same python3 -m dynamo.* commands in your Python environment. See Local Installation for host prerequisites and virtual environment setup.

For published NVIDIA container versions and tags, see Release Artifacts.

2

Start the frontend

Start the OpenAI-compatible frontend on port 8000:

$python3 -m dynamo.frontend --discovery-backend file

--discovery-backend file avoids needing etcd. To run the frontend and worker in the same terminal, background each command with > logfile.log 2>&1 &.

3

Start a worker

In another terminal, select the hardware and backend you installed, then launch the worker:

Choose your worker command

Select the same hardware and backend you used for the installation.

Hardware
Backend
Local worker
RunSGLang worker
NVIDIA GPU · Qwen/Qwen3-0.6B · file discovery
python3 -m dynamo.sglang --model-path Qwen/Qwen3-0.6B --discovery-backend file
4

Verify the endpoint

Check that the endpoint is up:

$curl -sf http://localhost:8000/health && echo OK

If you see OK, send a chat completion:

$curl localhost:8000/v1/chat/completions \
>  -H "Content-Type: application/json" \
>  -d '{"model": "Qwen/Qwen3-0.6B",
>       "messages": [{"role": "user", "content": "Hello!"}],
>       "max_tokens": 50}'

Connection refused? The frontend takes a few seconds to start — retry. For production liveness and readiness probes, see Health Check Reference.

From the Digest

Full-Stack Optimizations for Agentic Inference

How Dynamo optimizes for agentic workloads at three layers: the frontend API, the router, and KV cache management.

Flash Indexer: Inter-Galactic KV Routing

How Dynamo’s concurrent global index evolved through six iterations to sustain over 100M ops/sec.

Dive Deeper

Pick a full install path from the four options above, or explore how Dynamo works under the hood:

Architecture

How the frontend, router, and workers fit together.

Frontend Guide

Worker discovery, multi-model routing, OpenAI compat.

KV Cache Aware Routing

How the router places requests for prefix reuse.