Reinforcement Learning
Use Dynamo as the rollout-serving layer for RL workloads
Dynamo provides routing, worker management, weight-update controls, and serving telemetry for reinforcement learning (RL) rollouts. Your RL framework continues to own training, rewards, environments, trajectory semantics, checkpoints, and sample acceptance.
Use Dynamo when rollout serving has become a distributed-systems problem: many workers serve bursty traffic, samples share large prompt prefixes, policies must refresh without rebuilding the serving stack, or you need to diagnose and replay the serving workload independently of the trainer.
Proven on Real RL Workloads
In production, Cognition used Dynamo while training SWE-1.7 to manage inference-engine lifecycles and route inference across a multi-cluster RL system. When a replica failed, Dynamo rerouted inference to another worker and rescheduled the replica so the rollout pipeline could remain available while the latest policy state was restored.
Dynamo connects to these RL frameworks. verl publishes a Dynamo rollout recipe. NeMo RL includes a managed Dynamo generation backend. Prime-RL supports the Dynamo router as a drop-in routing option. The experimental Slime integration runs stock SGLang engines with Dynamo sidecars.
These integrations support different training frameworks and deployment models. The trainer retains ownership of RL semantics.
Keep Rollouts Running Through Failures
Dynamo detects worker loss and routes new work to healthy capacity. It can also migrate supported in-flight generations, reject new work explicitly when every eligible worker is overloaded, propagate client cancellation, and drain workers during planned shutdown. The RL framework still decides when to retry, discard, or accept a sample.
These serving behaviors were central to Cognition’s deployment above: failed replicas could be replaced without restarting the training system’s serving plane. Request migration and overload rejection are optional frontend behaviors; cancellation is built into supported request paths, while graceful shutdown and engine failover depend on the deployment. See Fault Tolerance for the supported behaviors, defaults, and limitations.
Decide Whether to Add Dynamo
Start with a specific bottleneck and a baseline metric. Measure cache reuse, worker imbalance, policy-refresh time, or time to diagnose a failure before adding Dynamo.
Where Dynamo Fits
Dynamo does not decide whether a trajectory is on-policy, accepted, or fresh enough for training. The framework must gate requests around synchronous updates or enforce its own bounded-staleness policy.
Framework Integrations
Experimental guides have runnable upstream artifacts but do not make a general compatibility promise. Integrations in progress remain in the compatibility table until a maintained path lands.
Kubernetes is optional for these RL integrations. Use it only when the selected framework or deployment environment requires it.