Deploying Dynamo Inference Graphs to Kubernetes#

This guide provides an overview of the different deployment options available for Dynamo inference graphs in Kubernetes environments.

Deployment Options#

Dynamo provides two distinct deployment paths, each serving different use cases:

1. 🚀 Dynamo Cloud Kubernetes Platform [PREFERRED]#

The Dynamo Cloud Platform (deploy/dynamo/helm/) provides a managed deployment experience:

  • Contains the infrastructure components required for the Dynamo cloud platform

  • Used when deploying with the dynamo deploy CLI commands

  • Provides a managed deployment experience

For detailed instructions on using the Dynamo Cloud Platform, see:

2. Manual Deployment with Helm Charts#

The manual deployment path (deploy/Kubernetes/) is available for users who need more control over their deployments:

  • Used for manually deploying inference graphs to Kubernetes

  • Contains Helm charts and configurations for deploying individual inference pipelines

  • Provides full control over deployment parameters

  • Requires manual management of infrastructure components

  • Documentation:

    • Manual Helm Deployment Guide: detailed instructions on manual deployment

    • [Deploying Dynamo Inference Graphs to Kubernetes using Helm](manual_helm_deployment.md#Deploying Dynamo Inference Graphs to Kubernetes using Helm): all-in-one script

Getting Started#

  1. For Dynamo Cloud Platform:

  2. For Manual Deployment:

Example Deployment#

See the [Hello World example](…/…/examples/hello_world.md#Deploying to and Running the Example in Kubernetes) for a complete walkthrough of deploying a simple inference graph.

See the LLM example for a complete walkthrough of deploying a production-ready LLM inference pipeline to Kubernetes.

On this page