Quickstart

This page is for version v0.8.1. For other versions, use one of these documentation indexes: - Latest (v1.5.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt - dev: https://docs.nvidia.com/dynamo/dev/llms.txt - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

# Install uv (recommended Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Create virtual environment and install Dynamo
uv venv venv
source venv/bin/activate
# Use prerelease flag to install RC versions of flashinfer and/or other dependencies
uv pip install --prerelease=allow "ai-dynamo[sglang]"  # or [vllm], [trtllm]

# Fetch and start etcd and NATS using Docker Compose
VERSION=$(uv pip show ai-dynamo | grep Version | cut -d' ' -f2)
curl -fsSL -o docker-compose.yml https://raw.githubusercontent.com/ai-dynamo/dynamo/refs/tags/v${VERSION}/deploy/docker-compose.yml
docker compose -f docker-compose.yml up -d

# Start the OpenAI compatible frontend (default port is 8000)
python -m dynamo.frontend
# In another terminal, start an SGLang worker
python -m dynamo.sglang --model-path Qwen/Qwen3-0.6B

curl localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-0.6B",
       "messages": [{"role": "user", "content": "Hello!"}],
       "max_tokens": 50}'