DocumentationThis directory contains practical examples demonstrating how to deploy and use Dynamo for distributed LLM inference. Each example includes setup instructions, configuration files, and explanations to help you understand different deployment patterns and use cases.
Want to see a specific example? Open a GitHub issue to request an example you’d like to see, or open a pull request if you’d like to contribute your own!
Learn fundamental Dynamo concepts through these introductory examples:
These examples show how Dynamo broadly works using major inference engines.
If you want to see advanced, framework-specific deployment patterns and best practices, check out the Examples Backends directory:
Platform-specific deployment guides for production environments:
Low-level runtime examples for developers using Python/Rust bindings:
Choose your deployment pattern: Start with the Quickstart for a simple local deployment, or explore Disaggregated Serving for advanced architectures.
Set up prerequisites: Most examples require etcd and NATS services. You can start them using:
Follow the example: Each directory contains detailed setup instructions and configuration files specific to that deployment pattern.
Before running any examples, ensure you have:
If you’re running Kubernetes/cloud deployment examples (EKS, AKS, GKE), you’ll also need:
See the Kubernetes Installation Guide for detailed setup instructions and pre-deployment checks.
