Local CLI User Guide | NVIDIA Dynamo Documentation

This page covers the local CLI path. For operator-managed deployments on a Kubernetes cluster, see the Kubernetes User Guide.

Jump Right In

Start the frontend and a model worker on one machine.

Serve a model with a frontend and worker, then explore larger deployment patterns.

Start the frontend and a model worker on one machine.

Install Dynamo and prepare the local services used by the frontend and workers.

Install from a container or Python package.

Use a local Kubernetes cluster when you need to test Kubernetes resources.

Configure local metrics and telemetry dependencies.

Start with an aggregated deployment, then add routing, disaggregation, multiple nodes, or KV cache offloading as needed.

Serve a model with a frontend and worker on one machine.

Route requests according to cached prompt prefixes.

Run prefill and decode as separate workers.

Run a CLI deployment across multiple machines.

Extend KV cache capacity beyond GPU memory.

Monitor the local deployment and expose health endpoints for automation.

Inspect metrics, logs, traces, request captures, and process health.