Reinforcement Learning
Use Dynamo as the rollout-serving layer for RL workloads
Dynamo provides routing, worker management, weight-update controls, and serving telemetry for reinforcement learning (RL) rollouts. Your RL framework continues to own training, rewards, environments, trajectory semantics, checkpoints, and sample acceptance.
Use Dynamo when rollout serving has become a distributed-systems problem: many workers serve bursty traffic, samples share large prompt prefixes, policies must refresh without rebuilding the serving stack, or you need to diagnose and replay the serving workload independently of the trainer.
Proven on Real RL Workloads
In production, Cognition used Dynamo while training SWE-1.7 to manage inference-engine lifecycles and route inference across a multi-cluster RL system. When a replica failed, Dynamo rerouted inference to another worker and rescheduled the replica so the rollout pipeline could remain available while the latest policy state was restored.
Dynamo also connects to a growing RL framework ecosystem. verl publishes a Dynamo rollout recipe, NeMo RL includes a managed Dynamo generation backend, and Prime-RL supports the Dynamo router as a drop-in routing option. Integration work with SLIME extends the same serving capabilities to SGLang-based rollout stacks. Together, these paths demonstrate that Dynamo can support different training frameworks and deployment models while leaving RL semantics with the trainer.
Keep Rollouts Running Through Failures
Dynamo detects worker loss and routes new work to healthy capacity. It can also migrate supported in-flight generations, reject new work explicitly when every eligible worker is overloaded, propagate client cancellation, and drain workers during planned shutdown. The RL framework still decides when to retry, discard, or accept a sample.
These serving behaviors were central to Cognition’s deployment above: failed replicas could be replaced without restarting the training system’s serving plane. Request migration and overload rejection are optional frontend behaviors; cancellation is built into supported request paths, while graceful shutdown and engine failover depend on the deployment. See Fault Tolerance for the supported behaviors, defaults, and limitations.
Decide Whether to Add Dynamo
Start with a specific bottleneck and a baseline metric. Measure cache reuse, worker imbalance, policy-refresh time, or time to diagnose a failure before adding Dynamo.
Where Dynamo Fits
Dynamo does not decide whether a trajectory is on-policy, accepted, or fresh enough for training. The framework must gate requests around synchronous updates or enforce its own bounded-staleness policy.
Framework Integrations
Experimental guides have runnable upstream artifacts but do not make a general compatibility promise. Integrations in progress remain in the compatibility table until a maintained path lands.
Kubernetes is optional for these RL integrations. Use it only when the selected framework or deployment environment requires it.
Next Steps
Route repeated rollout prefixes to cache-rich workers while accounting for live load and queue pressure.
Choose a trainer, artifact, or inference-peer source, then coordinate policy refresh and recovery.
Join framework records with Dynamo traces and metrics, inspect Perfetto timelines, and replay or simulate the serving workload.
Recover supported in-flight requests, shed overload explicitly, propagate cancellation, and drain workers safely.
Connect generation, worker administration, and policy updates while keeping RL semantics in your framework.