Local Guide | NVIDIA Dynamo Documentation

This page covers local deployments on a workstation or virtual machine. For operator-managed deployments on a Kubernetes cluster, see the Kubernetes Guide.

Jump Right In

Quickstart

Start the frontend and a model worker on one machine.

Model Deployment Overview

Serve a model with a frontend and worker, then explore larger deployment patterns.

Browse the Local Guide

Getting Started

Quickstart

Start the frontend and a model worker on one machine.

Installation

Install Dynamo and prepare the local services used by the frontend and workers.

Install Dynamo

Install from a container or Python package.

Minikube Setup

Use a local Kubernetes cluster when you need to test Kubernetes resources.

Model Deployment

Start with an aggregated deployment, then add routing, disaggregation, or KV cache offloading as needed.

Model Deployment Overview

Serve a model with a frontend and worker on one machine.

KV-Aware Routing

Route requests according to cached prompt prefixes.

Disaggregated Serving

Run prefill and decode as separate workers.

KV Cache Offloading

Extend KV cache capacity beyond GPU memory.

Operations

Monitor the local deployment and expose health endpoints for automation.

Observability

Inspect metrics, logs, traces, request captures, and process health.