Planner Examples

Planner-specific examples for advanced configuration and non-Kubernetes integrations. For DGDR manifests, see DGDR Templates. For the full configuration reference, see the Planner Guide.

Custom Load Predictors

Each YAML block in this section is a standalone PlannerConfig. Save the block as planner.yaml and pass it to python -m dynamo.planner --config planner.yaml. To use the same fields in a DGDR, nest them under spec.features.planner.

Warm-starting with Trace Data

Pre-load predictors with historical request patterns before live traffic:

1
optimization_target: sla
2
load_predictor: arima
3
load_predictor_warmup_trace: /data/trace.jsonl
4
load_predictor_log1p: true

The trace file should be in mooncake-style JSONL format with request-count, ISL, and OSL samples.

Kalman Filter Tuning

For workloads with rapid changes, tune the Kalman filter:

1
optimization_target: sla  # Required: predictor tuning is inert without it
2
load_predictor: kalman
3
kalman_q_level: 2.0       # Higher = more responsive to level changes
4
kalman_q_trend: 0.5       # Higher = trend changes faster
5
kalman_r: 5.0             # Lower = trusts new measurements more
6
kalman_min_points: 3      # Fewer points before forecasting starts
7
load_predictor_log1p: true  # Often helps with request-rate series

Prophet for Seasonal Workloads

For workloads with daily/weekly patterns:

1
optimization_target: sla  # Required: predictor tuning is inert without it
2
load_predictor: prophet
3
prophet_window_size: 100  # Larger window for seasonal detection
4
load_predictor_log1p: true

Virtual Connector

For non-Kubernetes environments, use the VirtualConnector to communicate scaling decisions:

1
from dynamo._core import DistributedRuntime, VirtualConnectorClient
2
3
# Initialize client
4
client = VirtualConnectorClient(distributed_runtime, namespace)
5
6
# Main loop: watch for planner decisions and execute them
7
while True:
8
    # Block until the planner makes a new scaling decision
9
    await client.wait()
10
11
    # Read the decision
12
    decision = await client.get()
13
    print(f"Scale to: prefill={decision.num_prefill_workers}, "
14
          f"decode={decision.num_decode_workers}, "
15
          f"id={decision.decision_id}")
16
17
    # Execute scaling in your environment
18
    scale_prefill_workers(decision.num_prefill_workers)
19
    scale_decode_workers(decision.num_decode_workers)
20
21
    # Report completion
22
    await client.complete(decision)

See components/planner/test/test_virtual_connector.py for a full working example.

Related Documentation

• Planner Guide — Planner configuration reference
• DGDR Templates — DGDR YAML examples
• Profiler Guide — Profiling workflow