> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt. # Planner Configuration (PlannerConfig) `PlannerConfig` is the configuration object for the Dynamo Planner. The Planner service parses it with [Pydantic](https://docs.pydantic.dev/), so every field, type, default, and constraint on this page comes from the [`PlannerConfig` model](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/planner/config/planner_config.py). For the deployment workflow and scaling-mode concepts, see the [Planner Guide](/dynamo/dev/knowledge-base/modular-components/planner/planner-guide); for the autoscaler overview, see the [Planner overview](/dynamo/dev/knowledge-base/modular-components/planner/overview). ## How the config is loaded The Planner service loads `PlannerConfig` from a single required `--config` argument — a path to a `.json`, `.yaml`, or `.yml` file, or an inline JSON string. The service auto-detects which, loads it, and validates it against the `PlannerConfig` model. A value that is neither a readable file nor valid JSON fails at startup. How you supply it depends on where you run the Planner. #### Kubernetes Set the object under `spec.features.planner` in a [DynamoGraphDeploymentRequest](/dynamo/dev/reference/api/kubernetes/dynamo-graph-deployment-request) (DGDR). DGDR passes it through without field-level validation, mounts it as a `planner-config-*` ConfigMap, and launches the Planner with `--config` pointing at the mounted file. The Planner validates it at startup. Every field below can be set here. ```yaml spec: features: planner: mode: disagg backend: vllm # optimization_target defaults to "throughput" — works with no further config ``` #### Local Run the Planner module directly and pass `--config` yourself, as either a file path or an inline JSON string: ```bash # From a YAML or JSON file python -m dynamo.planner --config planner_config.yaml # Or inline JSON python -m dynamo.planner --config '{"mode": "disagg", "backend": "vllm"}' ``` Several Prometheus fields default from environment variables and are excluded when the config is serialized back out. Set them either in the config object or through the environment variable noted on each field. See [Environment variables](#environment-variables). ## Core settings **`mode`** `string` — default: disagg Planner operating mode, matching the deployment topology. Allowed values: disagg prefill decode agg --- **`backend`** `string` — default: vllm Inference backend the Planner scales. Allowed values: vllm sglang trtllm mocker --- **`environment`** `string` — default: kubernetes Runtime environment that determines how the Planner applies scaling actions. Allowed values: kubernetes virtual global-planner --- **`namespace`** `string` — default: env DYN\_NAMESPACE, else dynamo Dynamo namespace of the deployment the Planner manages. Defaults from `DYN_NAMESPACE`, which the operator injects as `{k8s_namespace}-{dgd_name}`. Excluded from serialized output. --- **`model_name`** `string` — default: null Optional model name override. Auto-detected from the deployment when unset. --- **`global_planner_namespace`** `string` — default: null Namespace where the GlobalPlanner runs. **Required** when `environment` is `global-planner`. See the [Global Planner Guide](/dynamo/dev/knowledge-base/modular-components/planner/global-planner-guide). --- **`log_dir`** `string` — default: null Optional directory for Planner log output. --- ## Optimization target **`optimization_target`** `string` — default: throughput Scaling strategy. `throughput` and `latency` use static thresholds on queue depth and KV cache utilization — no SLA targets or profiling required. `load` uses user-defined prefill queue-token and decode KV-utilization thresholds. `sla` uses the Planner engine-query layer and AIC core performance model to target specific `ttft_ms` / `itl_ms` values. Allowed values: throughput latency load sla --- When `optimization_target` is `throughput`, `latency`, or `load`, the Planner forces load-based scaling on and throughput-based scaling off, and it ignores `ttft_ms` / `itl_ms`. The two scaling-mode flags below apply only when `optimization_target` is `sla`. **`enable_throughput_scaling`** `boolean` — default: true Enable predictive, traffic-based scaling. Only honored when `optimization_target` is `sla`. --- **`enable_load_scaling`** `boolean` — default: false Enable reactive, load-based scaling. Only honored when `optimization_target` is `sla`. At least one scaling mode must be enabled. --- **`ttft_ms`** `number` — default: 500.0 Time To First Token SLA target, in milliseconds. Also accepts the alias `ttft`. Must be greater than 0. Used only under `optimization_target: sla`. --- **`itl_ms`** `number` — default: 50.0 Inter-Token Latency SLA target, in milliseconds. Also accepts the alias `itl`. Used only under `optimization_target: sla`. --- ## Performance model and pre-deployment sweeping **`pre_deployment_sweeping_mode`** `string` — default: rapid How to generate optional bootstrap performance data. `none` skips bootstrap data; `rapid` uses AIConfigurator to simulate engine performance (\~30s); `thorough` measures on real GPUs (several hours). Allowed values: none rapid thorough --- **`profile_results_dir`** `string` — default: profiling\_results Directory holding profiler-generated performance data (npz or JSON), used to bootstrap or tune the performance model when the `get_perf_metrics` endpoint is unavailable. --- **`aic_perf_model`** `AICPerfModelSpec` — default: null Native AIConfigurator forward-pass model identity passed directly to the `aiconfigurator-core` wheel, enabling real-time AIC estimates with online correction. Unsupported native configs fall back to FPM regression. Does not trigger an AIC interpolation sweep. --- **`hf_id`** `string` — required HuggingFace model id, for example `Qwen/Qwen3-32B`. --- **`system`** `string` — required AIC system identifier, for example `h200_sxm`. --- **`backend`** `string` — required Allowed values: trtllm vllm sglang --- **`backend_version`** `string` — default: null Optional backend version string. --- **`prefill_pick`** `PickedParallelConfig` — default: null Parallelism pick for the prefill engine. Required for `mode` `disagg` or `prefill`. See [PickedParallelConfig](#pickedparallelconfig). --- **`decode_pick`** `PickedParallelConfig` — default: null Parallelism pick for the decode engine. Required for `mode` `disagg`, `decode`, or `agg`. See [PickedParallelConfig](#pickedparallelconfig). --- **`model_arch`** `string` — default: null Optional model architecture override. --- **`weight_dtype`** `string` — default: null Optional weight dtype. --- **`moe_dtype`** `string` — default: null Optional Mixture-of-Experts dtype. --- **`activation_dtype`** `string` — default: null Optional activation dtype. --- **`kv_cache_dtype`** `string` — default: null Optional KV cache dtype. --- **`aic_interpolation`** `AICInterpolationSpec` — default: null AIConfigurator interpolation spec. Populated by the profiler in rapid mode and written onto the Planner ConfigMap; you do not normally set this by hand. When present, the Planner runs the AIC sweep in-process at bootstrap to seed the performance model. See [AICInterpolationSpec](#aicinterpolationspec). --- ## GPU budget **`max_gpu_budget`** `integer` — default: 8 Maximum total GPUs the Planner may allocate across worker types. --- **`min_gpu_budget`** `integer` — default: -1 Per-DGD GPU floor enforced by the local Planner. `-1` disables it. When set with `max_gpu_budget` such that `min == max`, the Planner pins the per-DGD total and only redistributes replicas between prefill and decode. --- **`min_endpoint`** `integer` — default: 1 Minimum engine endpoints (replicas) for aggregated deployments. In disaggregated deployments, this value applies to both prefill and decode unless a role-specific value is set. In prefill-only or decode-only deployments, it supplies the active role when the corresponding role-specific value is unset. Must be nonnegative; use `0` for scale-to-zero compatibility. --- **`prefill_min_endpoint`** `integer` — default: null Minimum prefill endpoints in `disagg` and `prefill` modes. When set, replaces the prefill value supplied by `min_endpoint`. Must be at least `1`. --- **`decode_min_endpoint`** `integer` — default: null Minimum decode endpoints in `disagg` and `decode` modes. When set, replaces the decode value supplied by `min_endpoint`. Must be at least `1`. --- The Planner does not have per-component maximum endpoint fields. `max_gpu_budget`, the power budget when enabled, Global Planner allocation, and cluster capacity continue to bound scale-up. At startup and for runtime updates, the Planner rejects minimum endpoint combinations that cannot fit the configured GPU or power budget. **`decode_engine_num_gpu`** `integer` — default: null GPUs per decode engine replica. Auto-detected from the deployment when unset. --- **`prefill_engine_num_gpu`** `integer` — default: null GPUs per prefill engine replica. Auto-detected from the deployment when unset. --- ## Throughput-based scaling **`throughput_adjustment_interval_seconds`** `integer` — default: 180 Seconds between throughput-based scaling decisions. Also accepts the alias `throughput_adjustment_interval`. --- **`throughput_metrics_source`** `string` — default: frontend Prometheus traffic source. `frontend` reads `dynamo_frontend_*` metrics from the public Frontend; `router` reads `dynamo_component_router_*` from a LocalRouter (use for a pool-local Planner in GlobalPlanner deployments). Allowed values: frontend router --- ## Load-based scaling **`load_adjustment_interval_seconds`** `integer` — default: 5 Seconds between FPM tuning updates and load-based scaling decisions. Even when only throughput scaling is enabled, live FPM observations feed the performance model at this interval. Also accepts the alias `load_adjustment_interval`. Must be greater than 0 and, when both scaling modes are on, shorter than `throughput_adjustment_interval_seconds`. --- **`max_num_fpm_samples`** `integer` — default: 64 Maximum retained ForwardPassMetrics observations for online tuning and regression fallback. --- **`fpm_sample_bucket_size`** `integer` — default: 16 Number of buckets for observation retirement. Must be a perfect square. --- **`load_scaling_down_sensitivity`** `integer` — default: 80 Scale-down sensitivity from 0 to 100 (0 = never scale down, 100 = aggressive). --- **`load_min_observations`** `integer` — default: 5 Minimum observations before load-based scaling decisions begin (cold-start threshold). --- **`prefill_scale_up_queue_tokens`** `integer` — default: null Prefill queue-token count that triggers scale-up. Required (with the scale-down field) when `optimization_target` is `load` and `mode` includes prefill. Must be greater than 0 and greater than `prefill_scale_down_queue_tokens`. --- **`prefill_scale_down_queue_tokens`** `integer` — default: null Prefill queue-token count that allows scale-down. Required alongside `prefill_scale_up_queue_tokens`. --- **`decode_scale_up_kv_rate`** `number` — default: null Decode KV-utilization percentage (0–100) that triggers scale-up. Required (with the scale-down field) when `optimization_target` is `load` and `mode` includes decode. Must be greater than `decode_scale_down_kv_rate`. --- **`decode_scale_down_kv_rate`** `number` — default: null Decode KV-utilization percentage (0–100) that allows scale-down. Required alongside `decode_scale_up_kv_rate`. --- **`speculative_nextn`** `integer` — default: 0 Manual fallback speculative-decoding depth. A worker's published `runtime_config.runtime_data.spec_decode.nextn` takes precedence when present. Must be 0 or greater. --- ## Load prediction **`load_predictor`** `string` — default: arima Prediction method for request count, ISL, and OSL. Allowed values: constant arima kalman prophet --- **`load_predictor_log1p`** `boolean` — default: false Apply a log1p transform to predicted request count, ISL, and OSL. --- **`prophet_window_size`** `integer` — default: 50 Window size (seconds) for the Prophet predictor. --- **`load_predictor_warmup_trace`** `string` — default: null Path to a warmup trace file for bootstrapping predictions. --- **`kalman_q_level`** `number` — default: 1.0 Kalman process noise for the level component. --- **`kalman_q_trend`** `number` — default: 0.1 Kalman process noise for the trend component. --- **`kalman_r`** `number` — default: 10.0 Kalman measurement noise. --- **`kalman_min_points`** `integer` — default: 5 Minimum data points before Kalman predictions activate. --- ## Prometheus metrics These fields default from environment variables and are excluded from serialized output. Set them in the config object or through the noted environment variable. **`metric_pulling_prometheus_endpoint`** `string` — default: env PROMETHEUS\_ENDPOINT Prometheus endpoint the Planner queries for traffic metrics. Defaults to `PROMETHEUS_ENDPOINT`, else `http://prometheus-kube-prometheus-prometheus.monitoring.svc.cluster.local:9090`. --- **`metric_pulling_prometheus_token`** `string` — default: env PROMETHEUS\_TOKEN Optional bearer token sent as `Authorization: Bearer ` on every PromQL request. Read once at startup. --- **`metric_pulling_prometheus_token_file`** `string` — default: env PROMETHEUS\_TOKEN\_FILE Path to a file containing a bearer token, re-read before every request so rotated tokens are picked up without a restart. --- **`metric_pulling_prometheus_ssl_verify`** `boolean` — default: false Verify the upstream Prometheus TLS certificate. Defaults from `PROMETHEUS_SSL_VERIFY` (`1`/`true`/`yes` enable it). Pair with a CA bundle for a private CA. --- **`metric_pulling_prometheus_extra_query_params`** `object` — default: env PROMETHEUS\_EXTRA\_QUERY\_PARAMS Fixed key/value pairs appended as URL query parameters on every PromQL request. Set `PROMETHEUS_EXTRA_QUERY_PARAMS` as a URL query string, for example `namespace=my-ns&tenant=foo`. --- **`metric_pulling_prometheus_ca_bundle`** `string` — default: env PROMETHEUS\_CA\_BUNDLE Path to a CA bundle for verifying the upstream Prometheus TLS certificate. When set, the bundle is used for verification regardless of `metric_pulling_prometheus_ssl_verify`. Must point to an existing file. --- **`metric_reporting_prometheus_port`** `integer` — default: 0 Port on which the Planner exposes its own `dynamo_planner_*` metrics. Defaults from `PLANNER_PROMETHEUS_PORT`. `0` disables. --- ## Advisory mode **`advisory`** `boolean` — default: false Suggestion-only mode. The Planner computes, logs, exports, and reports recommended replica counts without executing scaling actions or changing the deployment. Use it to evaluate a new configuration or validate SLA targets against production traffic. --- ## Diagnostics and reporting **`report_interval_hours`** `number` — default: 24.0 Generate an HTML diagnostics report every N hours (simulated time). Set to `null` to disable. Must be a positive finite number or `null`. --- **`report_output_dir`** `string` — default: ./planner\_reports Directory for HTML diagnostics reports. --- **`report_filename`** `string` — default: null Fixed report filename written under `report_output_dir`. When unset, a timestamped name is used. --- **`report_write_gzip_log`** `boolean` — default: true Write a compressed JSONL diagnostics log (`.log.jsonl.gz`) next to each HTML report. --- **`live_dashboard_port`** `integer` — default: 8080 Port for the live diagnostics dashboard HTTP server. `0` disables. When enabled, visit `http://host:port/` for a real-time Plotly report. --- ## Runtime configuration **`control_api_port`** `integer` — default: 9086 Port for the runtime minimum-endpoint API. The server listens only on `127.0.0.1`; `0` disables it. Updates are process-local, are not persisted to the Planner ConfigMap, and take effect on the next planner tick. --- ### Runtime minimum endpoint API The Planner exposes `GET /v1/min-endpoints` and partial `PATCH /v1/min-endpoints`. The request and response fields depend on the active mode: | Mode | Runtime fields | | --------- | --------------------------------------------- | | `disagg` | `prefill_min_endpoint`, `decode_min_endpoint` | | `prefill` | `prefill_min_endpoint` | | `decode` | `decode_min_endpoint` | | `agg` | `min_endpoint` | The API has no authentication and is not exposed by a Kubernetes Service. For Kubernetes deployments, port-forward directly to the Planner pod: ```bash kubectl port-forward pod/ 9086:9086 curl http://127.0.0.1:9086/v1/min-endpoints curl --request PATCH http://127.0.0.1:9086/v1/min-endpoints \ --header 'Content-Type: application/json' \ --data '{"prefill_min_endpoint": 2}' ``` PATCH is atomic. Malformed JSON, unknown fields, and empty patches return HTTP `400`. `null`, a negative `min_endpoint`, component-specific minimums below `1`, fields that are inactive for the current mode, and minimums that exceed the configured GPU or power budget return HTTP `422` without changing the active configuration. ## Scheduling and plugin pipeline The Planner runs the builtin plugin pipeline by default. `scheduling` controls pipeline cadence; `plugin_registration` controls how plugins register, authenticate, and communicate. For the pipeline model, see the [Planner Guide](/dynamo/dev/knowledge-base/modular-components/planner/planner-guide#scheduling--plugin-pipeline). **`scheduling`** `SchedulingConfig` Plugin-pipeline scheduling. Rejects unknown fields. --- **`scale_interval_seconds`** `number` — default: gcd of enabled builtin intervals Base pipeline cadence. The pipeline wakes once per interval; each plugin's `execution_interval_seconds` decides whether it fires. When unset, computed as the gcd of `load_adjustment_interval_seconds` and, when throughput scaling is enabled, `throughput_adjustment_interval_seconds`. Must be greater than 0 and evenly divide both adjustment intervals. --- **`tick_max_duration_seconds`** `number` — default: 30.0 Outer deadline wrapping the full 4-stage pipeline. Exceeding it aborts the tick; the next tick runs from a clean state. Must be greater than 0. --- **`external_plugins`** `[]ExternalPluginEntry` — default: \[] Static external-plugin registrations applied at startup. Per-entry failures are logged but do not crash the Planner. See [ExternalPluginEntry](#externalpluginentry). --- **`gateway`** `GatewayConfig` gRPC registration gateway for self-registering plugins. Disabled by default. --- **`enabled`** `boolean` — default: false Open the gRPC gateway at `listen`. Not needed for static `external_plugins`. --- **`listen`** `string` — default: unix:///var/run/dynamo/planner/registry.sock Bind address. A `unix:` socket path for in-Pod registration, or `host:port` for TCP. --- **`allow_insecure`** `boolean` — default: false Permit binding a plaintext (no-TLS) gateway on a TCP listen. Fails closed by default because the gateway receives plugins' shared-secret tokens. `unix:` listens are always allowed. --- **`plugin_registration`** `PluginRegistrationConfig` Plugin registry configuration — auth, transport, heartbeat, and in-process plugins. Rejects unknown fields. --- **`auth`** `AuthConfig` Registry authentication. In the Planner process, an empty `trusted_sources` activates a legacy compatibility fallback to unauthenticated access and logs a warning. This fallback is **DEV ONLY**; do not rely on it as an intentional configuration. Select `allow_unauthenticated` explicitly for development. Production deployments must configure `static_secret`. --- **`trusted_sources`** `[]string` — default: \[] Accepted auth sources. Use `['allow_unauthenticated']` for development, or `['static_secret']` with `static_secrets` for production. Allowed values: static\_secret allow\_unauthenticated --- **`static_secrets`** `object` — default: \{} Map of `secret_value` to `subject_label`. Populate from a mounted Kubernetes Secret rather than hard-coding in the ConfigMap. --- **`transport`** `TransportConfig` Outbound gRPC transport settings for calling plugins. --- **`allow_insecure_grpc`** `boolean` — default: false Allow plaintext `grpc://` plugin endpoints. Required to use any `grpc://` endpoint; mTLS support is not yet shipped. --- **`request_timeout_seconds`** `number` — default: 5.0 Per-RPC timeout applied to every plugin call. Must be greater than 0. --- **`keepalive_time_ms`** `integer` — default: 30000 gRPC keepalive time, in milliseconds. --- **`max_message_size_bytes`** `integer` — default: 10485760 Maximum gRPC message size (10 MB). --- **`protocol_version_min`** `string` — default: 1.0 Minimum accepted plugin protocol version. --- **`protocol_version_max`** `string` — default: 1.0 Maximum accepted plugin protocol version. --- **`heartbeat_timeout_seconds`** `number` — default: 15.0 Reserved liveness setting. Heartbeats update registry timestamps, but no monitor currently applies this timeout or evicts plugins automatically. --- **`heartbeat_missed_threshold`** `integer` — default: 2 Reserved liveness setting. Automatic missed-heartbeat eviction is not currently wired. --- **`in_process_plugins`** `[]InProcessPluginSpec` — default: \[] In-process plugins loaded from Python modules at startup. See [InProcessPluginSpec](#inprocesspluginspec). --- **`admin`** `AdminAuthConfig` Admin (ListPlugins) RBAC config. Parsed but currently inert — the gateway default-denies `ListPlugins` regardless of `mode`. --- **`mode`** `string` — default: allow\_all Allowed values: allow\_all k8s\_rbac --- ## Additional types Nested object types referenced above, broken out to keep the field lists shallow. ### PickedParallelConfig A parallelism pick emitted by AIConfigurator. Referenced by `aic_perf_model.prefill_pick` / `decode_pick` and by `AICInterpolationSpec`. **`tp`** `integer` — default: 1 Tensor-parallel size. --- **`pp`** `integer` — default: 1 Pipeline-parallel size. --- **`dp`** `integer` — default: 1 Data-parallel (attention-DP) size. --- **`moe_tp`** `integer` — default: 1 Mixture-of-Experts tensor-parallel size. --- **`moe_ep`** `integer` — default: 1 Mixture-of-Experts expert-parallel size. --- ### AICInterpolationSpec Everything the Planner needs to reproduce a rapid-mode AIC sweep. Written by the profiler onto the Planner ConfigMap; not normally hand-authored. **`hf_id`** `string` — required HuggingFace model id. --- **`system`** `string` — required AIC system identifier. --- **`backend`** `string` — required Allowed values: trtllm vllm sglang --- **`isl`** `integer` — required Input sequence length for the sweep. Must be greater than 0. --- **`osl`** `integer` — required Output sequence length for the sweep. Must be greater than 0. --- **`sweep_max_context_length`** `integer` — required Maximum context length swept. Must be greater than 0. --- **`prefill_interpolation_granularity`** `integer` — required Number of prefill interpolation points. Must be greater than 0. --- **`decode_interpolation_granularity`** `integer` — required Number of decode interpolation points. Must be greater than 0. --- **`prefill_pick`** `PickedParallelConfig` — required Prefill parallelism pick. See [PickedParallelConfig](#pickedparallelconfig). --- **`decode_pick`** `PickedParallelConfig` — required Decode parallelism pick. See [PickedParallelConfig](#pickedparallelconfig). --- ### ExternalPluginEntry One static external-plugin registration under `scheduling.external_plugins`. Rejects unknown fields. **`plugin_id`** `string` — required Unique identifier. Must not collide with a builtin plugin id. --- **`plugin_type`** `string` — required Pipeline stage the plugin participates in. Allowed values: predict propose reconcile constrain --- **`priority`** `integer` — required Stage priority. Smaller number is more authoritative. --- **`endpoint`** `string` — required Wire endpoint. Must start with `grpc://host:port`; `inproc://` is rejected here. --- **`auth_token`** `string` Bearer token validated by the registry. Populate from a mounted Secret. --- **`protocol_version`** `string` — default: 1.0 Plugin protocol version. Must fall within the registry's supported range. --- **`version`** `string` — default: v1 Plugin's own version string, surfaced in ListPlugins. --- **`execution_interval_seconds`** `number` — default: 0.0 `0.0` runs every tick; a positive value throttles to every N seconds. Must be 0 or greater. --- **`hold_policy`** `string` — default: HOLD\_LAST Behavior when throttled. `HOLD_LAST` reuses the cached result; `ACCEPT_WHEN_IDLE` treats the plugin as no-opinion when not due. Accepts the name (case-insensitive) or its integer value. Allowed values: HOLD\_LAST ACCEPT\_WHEN\_IDLE --- **`needs`** `[]string` — default: \[] Capability list consumed by the type-aware merge. --- **`requires_produced_fields`** `[]string` — default: \[] Dot-paths into the pipeline context (for example `predictions`, `observations.traffic`) that must be set for the plugin to fire on a tick. Empty means no gating. --- **`observation_window_seconds`** `number` — default: 0.0 Aggregation window for windowed observation types in `needs`. `0.0` uses `scale_interval` freshness; a positive value aggregates over the last N seconds and must be an integer multiple of `scale_interval_seconds`. --- ### InProcessPluginSpec One in-process plugin under `plugin_registration.in_process_plugins`. Rejects unknown fields. **`module`** `string` — required Python module path containing the plugin class. --- **`class`** `string` — required Plugin class name within `module`. --- **`plugin_id`** `string` — required Unique plugin identifier. --- **`plugin_type`** `string` — required Allowed values: predict propose reconcile constrain --- **`priority`** `integer` — required Stage priority. Smaller number is more authoritative. --- **`execution_interval_seconds`** `number` — default: 0.0 `0.0` runs every tick; a positive value throttles to every N seconds. --- **`hold_policy`** `string` — default: ACCEPT\_WHEN\_IDLE Behavior when throttled. Allowed values: ACCEPT\_WHEN\_IDLE HOLD\_LAST --- **`needs`** `[]string` — default: \[] Capability list consumed by the type-aware merge. --- **`requires_produced_fields`** `[]string` — default: \[] Dot-paths into the pipeline context that must be set for the plugin to fire on a tick. --- **`observation_window_seconds`** `number` — default: 0.0 Aggregation window for windowed observation types in `needs`. Must be `0.0` or a positive integer multiple of `scale_interval_seconds`. --- **`kwargs`** `object` — default: \{} Keyword arguments passed to the plugin class constructor. --- ## Environment variables The Planner reads these environment variables. The Prometheus and namespace variables set the defaults for the config fields noted above; the rest control runtime behavior for specific environments. | Variable | Effect | Default | | ------------------------------- | --------------------------------------------------------------------------------------- | -------------- | | `DYN_NAMESPACE` | Default for `namespace`. Operator injects `{k8s_namespace}-{dgd_name}`. | `dynamo` | | `PROMETHEUS_ENDPOINT` | Default for `metric_pulling_prometheus_endpoint`. | in-cluster URL | | `PROMETHEUS_TOKEN` | Default for `metric_pulling_prometheus_token`. | unset | | `PROMETHEUS_TOKEN_FILE` | Default for `metric_pulling_prometheus_token_file`. | unset | | `PROMETHEUS_SSL_VERIFY` | Default for `metric_pulling_prometheus_ssl_verify` (`1`/`true`/`yes`). | `false` | | `PROMETHEUS_EXTRA_QUERY_PARAMS` | Default for `metric_pulling_prometheus_extra_query_params` (URL query string). | unset | | `PROMETHEUS_CA_BUNDLE` | Default for `metric_pulling_prometheus_ca_bundle`. | unset | | `PLANNER_PROMETHEUS_PORT` | Default for `metric_reporting_prometheus_port`. | `0` | | `DYN_PARENT_DGD_K8S_NAME` | Parent DGD name for the GlobalPlanner connector (`environment: global-planner`). | required | | `POD_NAMESPACE` | Kubernetes namespace for the GlobalPlanner connector. | required | | `SCALING_CHECK_INTERVAL` | Poll interval (seconds) for the virtual environment connector (`environment: virtual`). | `10` | | `SCALING_MAX_WAIT_TIME` | Maximum wait (seconds) for a scaling action in the virtual environment. | `1800` | | `DYNAMO_PLANNER_TEST` | Test-only. Must be `1` to allow the virtual clock. Not for production. | unset | ## Validation rules The Planner enforces these cross-field rules at startup and rejects a config that violates any of them: * `ttft_ms` must be greater than 0. * `report_interval_hours` must be a positive finite number or `null`. * `fpm_sample_bucket_size` must be a perfect square. * `global_planner_namespace` is required when `environment` is `global-planner`. * Under `optimization_target: load`, prefill modes require `prefill_scale_up_queue_tokens` > `prefill_scale_down_queue_tokens`, and decode modes require `decode_scale_up_kv_rate` > `decode_scale_down_kv_rate`. * At least one scaling mode must be enabled. Under any `optimization_target` other than `sla`, the Planner forces load scaling on and throughput scaling off and ignores `ttft_ms` / `itl_ms`. * When `aic_perf_model` is set, `prefill_pick` is required for `mode` `disagg`/`prefill` and `decode_pick` for `mode` `disagg`/`decode`/`agg`. * `scheduling.scale_interval_seconds` must evenly divide `load_adjustment_interval_seconds` and, when throughput scaling is enabled, `throughput_adjustment_interval_seconds`. * When both scaling modes are enabled, `load_adjustment_interval_seconds` must be shorter than `throughput_adjustment_interval_seconds`. ## Related pages #### [Planner Guide](/dynamo/dev/knowledge-base/modular-components/planner/planner-guide) Deployment workflow, scaling modes, and profiler integration. #### [Planner Overview](/dynamo/dev/knowledge-base/modular-components/planner/overview) Why LLM inference needs a different autoscaler. #### [DGDR Reference](/dynamo/dev/reference/api/kubernetes/dynamo-graph-deployment-request) The custom resource whose `spec.features.planner` carries this config. #### [Environment Variables](/dynamo/dev/reference/observability/environment-variables) Observability environment variables shared across Dynamo processes.