Model Deployment
A DynamoGraphDeployment (DGD) is the Kubernetes entry point for serving a model with Dynamo. It defines the frontend and model workers, their images and resources, and how requests flow through the inference graph. When you apply a DGD, the Dynamo operator creates and manages the underlying Kubernetes workloads and services.
The DGD is the deployment artifact that ultimately serves traffic, regardless of whether you start from a tuned manifest, copy a template, generate one, or write the resource directly.
Create a DGD
Choose the starting point that best matches your model, hardware, and desired level of control.
Use a tuned DGD manifest when a recipe matches your model, backend, GPU, and serving topology.
Start from an aggregated, disaggregated, or multinode manifest for vLLM, SGLang, or TensorRT-LLM.
Learn when to use DGDR and how the Profiler and Planner generate and operate a DGD.
Define components, parallelism, replicas, and Kubernetes settings for full control over the deployment.
Choose a Deployment Topology
Start with aggregated serving when one worker can handle both prompt processing and token generation. For more demanding models and workloads, use these guides to split, size, or distribute the deployment.
Run prefill and decode in separate worker pools so you can size and scale each phase independently.
Evaluate parallelism and replica configurations against your hardware and latency target.
Run workers that require more GPUs than a single Kubernetes node provides.