dynamo.planner

Raised when the backend framework does not exist.

1
from dynamo.planner.errors import BackendFrameworkInvalidError

1
BackendFrameworkInvalidError(backend_framework: str)

This occurs when the DynamoGraphDeployment contains an unsupported backend framework.

components/src/dynamo/planner/errors.py#L155

Public methods

init

1
__init__(backend_framework: str)

No summary available.

source

Raised when the backend framework is not supported.

1
from dynamo.planner.errors import BackendFrameworkNotFoundError

1
BackendFrameworkNotFoundError()

This occurs when the DynamoGraphDeployment contains an unsupported backend framework.

components/src/dynamo/planner/errors.py#L144

Public methods

init

1
__init__()

No summary available.

source

Base class for component type configuration issues.

1
from dynamo.planner.errors import ComponentError

This serves as a parent class for all exceptions related to component type configuration problems in DynamoGraphDeployments.

components/src/dynamo/planner/errors.py#L99

Raised when the model name is not the same in the deployment

1
from dynamo.planner.errors import DeploymentModelNameMismatchError

1
DeploymentModelNameMismatchError(prefill_model_name: str, decode_model_name: str)

components/src/dynamo/planner/errors.py#L117

Public methods

init

1
__init__(prefill_model_name: str, decode_model_name: str)

No summary available.

source

Raised when deployment validation fails for multiple components.

1
from dynamo.planner.errors import DeploymentValidationError

1
DeploymentValidationError(errors: List[str])

This is used to aggregate multiple validation errors into a single exception, providing a comprehensive view of all validation issues.

components/src/dynamo/planner/errors.py#L214

Public methods

init

1
__init__(errors: List[str])

No summary available.

source

Raised when multiple components have the same planner role.

1
from dynamo.planner.errors import DuplicateSubComponentError

1
DuplicateSubComponentError(sub_component_type: str, service_names: List[str])

This occurs when the DynamoGraphDeployment contains more than one component with the same role, which violates the expected uniqueness constraint.

components/src/dynamo/planner/errors.py#L191

Public methods

init

1
__init__(sub_component_type: str, service_names: List[str])

No summary available.

source

Raised when Parent DynamoGraphDeployment cannot be found.

1
from dynamo.planner.errors import DynamoGraphDeploymentNotFoundError

1
DynamoGraphDeploymentNotFoundError(deployment_name: str, namespace: str)

This typically occurs when:

• The DYN_PARENT_DGD_K8S_NAME environment variable is not set
• The referenced DynamoGraphDeployment doesn’t exist in the namespace

components/src/dynamo/planner/errors.py#L55

Public methods

init

1
__init__(deployment_name: str, namespace: str)

No summary available.

source

Raised when a DynamoGraphDeployment is not ready for scaling.

1
from dynamo.planner.errors import DynamoGraphDeploymentNotReadyError

1
DynamoGraphDeploymentNotReadyError(deployment_name: str, namespace: str | None = None)

components/src/dynamo/planner/errors.py#L75

Public methods

init

1
__init__(deployment_name: str, namespace: str | None = None)

No summary available.

source

Planner search-space preparation and replay-spec materialization.

1
from dynamo.planner.simulation import DynamoPlannerSweepConfigProvider

components/src/dynamo/planner/simulation/provider.py#L256

Public methods

generate_search_space

1
generate_search_space(search_spec: Mapping[str, JSONValue], context: SweepContext) -> AdapterSearchPlan

No summary available.

source

materialize_replay

1
materialize_replay(plan: AdapterSearchPlan, selection: Mapping[str, JSONValue], context: CandidateContext) -> AdapterReplaySpec

No summary available.

source

Raised when target_replicas is empty or invalid.

1
from dynamo.planner.errors import EmptyTargetReplicasError

1
EmptyTargetReplicasError()

This occurs when attempting to set component replicas with an empty or invalid target_replicas dictionary.

components/src/dynamo/planner/errors.py#L227

Public methods

init

1
__init__()

No summary available.

source

Static capabilities for a single engine stage (prefill or decode).

1
from dynamo.planner.core import EngineCapabilities

1
EngineCapabilities(num_gpu: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, max_num_seqs: Optional[int] = None, context_length: Optional[int] = None, max_kv_tokens: Optional[int] = None, kv_cache_block_size: Optional[int] = None, speculative_nextn: Optional[int] = None, power_watts_per_replica: Optional[int] = None) -> None

components/src/dynamo/planner/core/types.py#L197

Public methods

init

1
__init__(num_gpu: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, max_num_seqs: Optional[int] = None, context_length: Optional[int] = None, max_kv_tokens: Optional[int] = None, kv_cache_block_size: Optional[int] = None, speculative_nextn: Optional[int] = None, power_watts_per_replica: Optional[int] = None) -> None

No summary available.

source

Per-engine ForwardPassMetrics keyed by (worker_id, dp_rank).

1
from dynamo.planner.core import FpmObservations

1
FpmObservations(prefill: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None, decode: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None) -> None

components/src/dynamo/planner/core/types.py#L74

Public methods

init

1
__init__(prefill: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None, decode: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None) -> None

No summary available.

source

Connector that delegates scaling decisions to a centralized GlobalPlanner.

1
from dynamo.planner import GlobalPlannerConnector

1
GlobalPlannerConnector(runtime: DistributedRuntime, dynamo_namespace: str, global_planner_namespace: str, global_planner_component: str = 'GlobalPlanner', model_name: Optional[str] = None)

This connector wraps RemotePlannerClient and implements the InfraScaler interface, allowing planner_core.py to treat global-planner environment mode consistently with kubernetes and virtual modes.

components/src/dynamo/planner/connectors/global_planner.py#L37

Public methods

init

1
__init__(runtime: DistributedRuntime, dynamo_namespace: str, global_planner_namespace: str, global_planner_component: str = 'GlobalPlanner', model_name: Optional[str] = None)

Initialize GlobalPlannerConnector.

Parameters

source

async_init

1
async_init()

Async initialization - creates RemotePlannerClient

source

set_predicted_load

1
set_predicted_load(num_requests: Optional[float], isl: Optional[float], osl: Optional[float])

Set predicted load for inclusion in next scale request.

This is called by planner_core.py before calling set_component_replicas.

source

set_component_replicas

1
set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set component replicas by delegating to GlobalPlanner.

Sends a ScaleRequest to the GlobalPlanner with the target replica configurations.

Parameters

Raises

• EmptyTargetReplicasError — If target_replicas is empty
• RuntimeError — If remote_client is not initialized or the response indicates a hard error (e.g., authorization denied, K8s exception). A REJECTED response is NOT raised — it is logged as a warning and treated as a no-op for this tick.

source

add_component

1
add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component (not supported for GlobalPlanner).

GlobalPlanner only supports batch operations via set_component_replicas.

source

remove_component

1
remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component (not supported for GlobalPlanner).

GlobalPlanner only supports batch operations via set_component_replicas.

source

validate_deployment

1
validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True) -> None

Validate deployment (no-op for GlobalPlanner).

The GlobalPlanner validates the deployment on its side, so local validation is not needed in delegating mode.

source

wait_for_deployment_ready

1
wait_for_deployment_ready(include_planner: bool = True)

Wait for the pool’s own workers to be ready.

Even though GlobalPlanner handles cluster-wide orchestration, the pool Planner still reads its own workers’ DynamoWorkerMetadata CRs for capability discovery (get_worker_info). Without a local wait, _async_init runs within milliseconds of pod entry — long before workers register MDC — so get_worker_info falls back to defaults with context_length / max_kv_tokens unset and load-scaling silently disables itself for the pod’s lifetime.

Mirror the standalone path by delegating to the pool-local KubernetesConnector. If no local connector is available (e.g. running outside a cluster), fall back to the previous no-op so out-of-cluster callers are not blocked.

source

get_actual_worker_counts

1
get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Read ready replica counts and rollout stability from the pool’s own DGD.

GlobalPlanner orchestrates scaling, but the pool Planner pod has direct access to its own DGD status. Mirror KubernetesConnector by delegating to the pool-local connector so _scaling_in_progress observes real rollouts instead of always seeing is_stable=True.

Returns (0, 0, True) when no local KubernetesConnector is available (e.g. running out-of-cluster), matching the existing capability-discovery fallback path so out-of-cluster callers aren’t blocked.

source

get_worker_runtime_namespace

1
get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

Resolve the pool-local worker runtime namespace when available.

source

get_gpu_counts

1
get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

Resolve pool-local GPU shape when available.

source

get_worker_info

1
get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

Resolve per-worker capabilities from the pool’s own MDC/DGD.

Without this, resolve_worker_info falls through to build_worker_info_from_defaults which leaves context_length and max_kv_tokens unset, and load_scaling’s easy-mode decisions bail out every tick — so the pool Planner silently sends no ScaleRequests.

source

get_model_name

1
get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get model name.

Prefers the value provided at init time, then the pool’s own DGD container args (via the local KubernetesConnector), and finally falls back to a placeholder indicating the model is managed remotely.

source

No summary available.

1
from dynamo.planner import KubernetesConnector

1
KubernetesConnector(dynamo_namespace: str, model_name: Optional[str] = None, k8s_namespace: Optional[str] = None, parent_dgd_name: Optional[str] = None, raise_not_ready: bool = False)

components/src/dynamo/planner/connectors/kubernetes.py#L66

Public methods

init

1
__init__(dynamo_namespace: str, model_name: Optional[str] = None, k8s_namespace: Optional[str] = None, parent_dgd_name: Optional[str] = None, raise_not_ready: bool = False)

No summary available.

source

async_init

1
async_init()

No-op asynchronous lifecycle hook.

source

get_worker_runtime_namespace

1
get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

Return the Dynamo namespace used by the current worker generation.

Newer operators publish the effective runtime namespace on the worker component status. Older operators expose only the active worker hash, so the planner falls back to appending that hash only for Deployment-backed and LeaderWorkerSet-backed workers.

source

add_component

1
add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component by increasing its replica count by 1

source

remove_component

1
remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component by decreasing its replica count by 1

source

validate_deployment

1
validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True)

Verify that the deployment contains prefill/decode components and the model name exists. Allows explicit component-name overrides when the caller provides them.

Raises

• DynamoGraphDeploymentNotFoundError — If the deployment is not found
• DeploymentValidationError — If the deployment does not contain required prefill/decode components

source

get_model_name

1
get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get the model name from the current deployment.

source

get_graph_deployment

1
get_graph_deployment() -> dict

Fetch the DGD once for callers that share it across GPU/power reads.

Not on the base PlannerConnector protocol — power awareness is Kubernetes-local and must not expand that ABC. The environment checks is_power_aware_connector(controller) (all four methods present) rather than duck-typing via getattr.

source

get_gpu_counts

1
get_gpu_counts(require_prefill: bool = True, require_decode: bool = True, deployment: Optional[dict] = None) -> tuple[int, int]

Get the GPU counts for prefill and decode components.

Pass deployment to reuse an already-fetched DGD (avoids a second GET when the environment also resolves power configs on the same tick).

source

get_component_power_configs

1
get_component_power_configs(require_prefill: bool = True, require_decode: bool = True, prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, deployment: Optional[dict] = None) -> tuple[Optional[ComponentPowerConfig], Optional[ComponentPowerConfig]]

Resolve DGD-owned per-role power configs from worker podTemplate annotations.

One DGD GET unless deployment is provided (shared with get_gpu_counts on the same tick). watts_per_replica on each config uses the replica-wide GPU total (nodeCount × per-pod) via Service.get_total_gpu_count(), independent of the per-pod num_gpus the GPU-budget math consumes.

The typed parser errors (PowerAnnotationMissingError / PowerAnnotationInvalidError / SubComponentNotFoundError / DuplicateSubComponentError / ValueError for a bad GPU count) propagate so the environment can apply the startup-fail vs runtime-conservative policy rather than the planner guessing a cap.

source

get_frontend_metrics_url

1
get_frontend_metrics_url(port: int = 8000) -> Optional[str]

Auto-discover the frontend component’s metrics URL from the DGD.

Iterates DGD components to find the component with type “frontend”, then constructs the in-cluster URL using the operator’s naming convention: http://{dgd_name}-{component_name_lowercase}:{port}/metrics

Returns

• Optional[str] — The metrics URL string, or None if no frontend component is found.

source

wait_for_deployment_ready

1
wait_for_deployment_ready(include_planner: bool = True)

Wait for the deployment to be ready (legacy replica-stability path).

Does not check pod annotation convergence or require observedGeneration catch-up. Power-aware callers that permanently cache DGD fields must use wait_for_settled_graph_deployment instead.

Parameters

source

wait_for_settled_graph_deployment

1
wait_for_settled_graph_deployment(include_planner: bool = False, *, require_prefill: bool = True, require_decode: bool = True, prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> dict

Wait for a settled DGD snapshot and return that same object.

When include_planner is False, the snapshot has:

• non-planner worker replica counts stable (desired == updated == ready)
• status.observedGeneration >= metadata.generation
• every non-terminal worker Pod carries the expected dynamo.nvidia.com/gpu-power-limit annotation from the current DGD snapshot, confirming the operator has propagated the DGD intent to running Pods (hardware enforcement by the Power Agent/NVML is separate and not verified here)

Power-relevant workers are selected with the same role/name resolution as get_component_power_configs (typed roles, explicit-name fallback for untyped workers, unique generic type: worker for agg).

Callers that permanently cache fields from the DGD (power caps) must use this snapshot rather than issuing a later GET, so an annotation-only generation bump cannot be adopted before workers have rolled onto that generation. Active rolling updates (status.rollingUpdate.phase Pending/InProgress/Failed) also block settlement because old Pods still carry the previous cap.

source

get_worker_info

1
get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

Get WorkerInfo for a sub-component, trying MDC first, then fallbacks.

Parameters

source

get_actual_worker_counts

1
get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Get ready worker counts from DGD status without listing Pods.

source

get_power_aware_worker_counts

1
get_power_aware_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Get power-safe worker counts without blocking the Planner event loop.

One thread dispatch contains the synchronous DGD GET and the single DGD-scoped Pod LIST. The returned Pod snapshot is partitioned locally by component before terminating-Pod checks run.

source

set_component_replicas

1
set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set the replicas for multiple components at once

source

Raised when the model name is not found in the deployment

1
from dynamo.planner.errors import ModelNameNotFoundError

1
ModelNameNotFoundError()

components/src/dynamo/planner/errors.py#L109

Public methods

init

1
__init__()

No summary available.

source

Deployment-control interface the planner uses to inspect and scale one deployment.

1
from dynamo.planner import PlannerConnector

construct_connector selects one implementation per PlannerConfig.environment: KubernetesConnector scales through the DGD scaling adapter’s Scale subresource and falls back to patching DGD replica counts directly when no adapter exists, VirtualConnector publishes decisions through the runtime coordinator for the deployment environment to apply, and GlobalPlannerConnector forwards them to a centralized GlobalPlanner. PlannerEnvironmentImpl.initialize drives async_init, then validate_deployment, then wait_for_deployment_ready; async_init has to run first, because GlobalPlannerConnector.set_component_replicas raises RuntimeError until it holds a remote client.

A clean return is not proof of the outcome. validate_deployment inspects the deployment only under Kubernetes and is a no-op in the other two modes. get_gpu_counts yields (None, None) from VirtualConnector always and from GlobalPlannerConnector when it holds no pool-local Kubernetes connector, while the Kubernetes implementation narrows the return to tuple[int, int] and raises DeploymentValidationError rather than reporting an unknown shape. get_model_name can return the placeholder "managed-remotely" under a global planner. set_component_replicas may log and return without scaling when the deployment is not ready or the global planner rejects the request, though all three implementations do raise EmptyTargetReplicasError on an empty target list. get_actual_worker_counts reports 0 for a component whose name argument is None rather than a deployment-wide total.

Being a Protocol rather than an ABC, every method body here is pass. All three implementations subclass it explicitly, so an override that is missing or misnamed returns None at runtime instead of raising. The surface callers rely on is also wider than what is declared here: construct_environment feature-detects get_worker_runtime_namespace, which all three connectors provide.

components/src/dynamo/planner/connectors/base.py#L32

Public methods

async_init

1
async_init() -> None

No summary available.

source

validate_deployment

1
validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True) -> None

No summary available.

source

wait_for_deployment_ready

1
wait_for_deployment_ready(include_planner: bool = True) -> None

No summary available.

source

get_model_name

1
get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

No summary available.

source

get_gpu_counts

1
get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

No summary available.

source

get_actual_worker_counts

1
get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

No summary available.

source

set_component_replicas

1
set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True) -> None

No summary available.

source

What the core returns after processing a tick.

1
from dynamo.planner.core import PlannerEffects

1
PlannerEffects(scale_to: Optional[ScalingDecision] = None, next_tick: Optional[ScheduledTick] = None, diagnostics: TickDiagnostics = TickDiagnostics()) -> None

components/src/dynamo/planner/core/types.py#L188

Public methods

init

1
__init__(scale_to: Optional[ScalingDecision] = None, next_tick: Optional[ScheduledTick] = None, diagnostics: TickDiagnostics = TickDiagnostics()) -> None

No summary available.

source

Base exception for all planner-related errors.

1
from dynamo.planner.errors import PlannerError

This serves as the root exception class for all custom exceptions in the planner module, allowing for broad exception catching when needed.

components/src/dynamo/planner/errors.py#L45

Shared in-memory scaling state for all planner modes.

1
from dynamo.planner.core import PlannerScalingState

1
PlannerScalingState(config: PlannerConfig, capabilities: Optional[WorkerCapabilities] = None) -> None

Owns perf models, throughput lower bounds, worker inventory, last-value runtime metadata, and all scaling decision logic. It deliberately has no runtime dependencies. Load prediction state lives in the builtin PREDICT plugin and is passed in explicitly.

Builtin orchestrator plugins use this class directly as their private shared core while the remaining cross-plugin state is being split into explicit pipeline artifacts.

components/src/dynamo/planner/core/state_machine.py#L48

Public methods

init

1
__init__(config: PlannerConfig, capabilities: Optional[WorkerCapabilities] = None) -> None

No summary available.

source

update_capabilities

1
update_capabilities(capabilities: WorkerCapabilities) -> None

Replace the current worker capabilities.

source

load_benchmark_fpms

1
load_benchmark_fpms(prefill_fpms: Optional[list[ForwardPassMetrics]] = None, decode_fpms: Optional[list[ForwardPassMetrics]] = None, agg_fpms: Optional[list[ForwardPassMetrics]] = None) -> None

No summary available.

source

begin_tick

1
begin_tick() -> None

Reset per-tick diagnostics before builtin plugins run.

source

observe_worker_counts

1
observe_worker_counts(counts: WorkerCounts) -> None

No summary available.

source

observe_fpm

1
observe_fpm(obs: FpmObservations) -> None

No summary available.

source

observe_runtime_metadata

1
observe_runtime_metadata(*, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

Update last-value runtime metadata without touching prediction history.

source

install_regressions

1
install_regressions(*, prefill: Optional[PlannerEnginePerfModel] = None, decode: Optional[PlannerEnginePerfModel] = None, agg: Optional[PlannerEnginePerfModel] = None) -> None

No summary available.

source

advance_load

1
advance_load(obs: FpmObservations, *, predicted_kv_hit_rate: Optional[float] = None, predicted_accept_length: Optional[float] = None) -> Optional[ScalingDecision]

No summary available.

source

advance_throughput_from_prediction

1
advance_throughput_from_prediction(traffic: TrafficObservation, *, predicted_num_req: Optional[float], predicted_isl: Optional[float], predicted_osl: Optional[float], predicted_kv_hit_rate: Optional[float], predicted_accept_length: Optional[float] = None) -> Optional[ScalingDecision]

Run the throughput decision using PREDICT-stage output.

The PREDICT plugin owns load prediction history. This method consumes only explicit prediction output so PROPOSE never re-runs prediction or depends on hidden predictor state.

source

diagnostics

1
diagnostics() -> TickDiagnostics

No summary available.

source

Raised when a per-GPU power-limit annotation is not a positive integer.

1
from dynamo.planner.errors import PowerAnnotationInvalidError

1
PowerAnnotationInvalidError(component_name: str, value: str)

The annotation value is watts and must parse to an integer > 0. Empty, non-numeric, zero, or negative values are configuration errors — the planner cannot compute a power budget from them and refuses to silently substitute a default.

components/src/dynamo/planner/errors.py#L266

Public methods

init

1
__init__(component_name: str, value: str)

No summary available.

source

Raised when a worker component has no per-GPU power-limit annotation.

1
from dynamo.planner.errors import PowerAnnotationMissingError

1
PowerAnnotationMissingError(component_name: str)

The per-GPU cap is authored on the worker component’s podTemplate.metadata.annotations (key dynamo.nvidia.com/gpu-power-limit); a missing key means the DGD did not declare a cap for this role. When power awareness is enabled every managed worker role must carry one, so the planner fails closed rather than guessing a cap.

components/src/dynamo/planner/errors.py#L241

Public methods

init

1
__init__(component_name: str)

No summary available.

source

Raised when the operator marks a DGD rolling update as Failed.

1
from dynamo.planner.errors import RolloutFailedError

1
RolloutFailedError(deployment_name: str, reason: str = '')

Failed is a terminal rollout state (the operator sets endTime). Retrying until a generic timeout would leave the planner stuck for up to 30 minutes; instead we surface this as an immediate actionable error so the operator or user can investigate and recover.

components/src/dynamo/planner/errors.py#L292

Public methods

init

1
__init__(deployment_name: str, reason: str = '')

No summary available.

source

No summary available.

1
from dynamo.planner import SLAPlannerDefaults

components/src/dynamo/planner/config/defaults.py#L44

Desired replica counts. None means the core has no opinion on that component (e.g. prefill-only planner leaves decode as None).

1
from dynamo.planner.core import ScalingDecision

1
ScalingDecision(num_prefill: Optional[int] = None, num_decode: Optional[int] = None) -> None

components/src/dynamo/planner/core/types.py#L96

Public methods

init

1
__init__(num_prefill: Optional[int] = None, num_decode: Optional[int] = None) -> None

No summary available.

source

Declares when the core next needs to be called, what data it needs, and what decisions to make.

1
from dynamo.planner.core import ScheduledTick

1
ScheduledTick(at_s: float, run_load_scaling: bool = False, run_throughput_scaling: bool = False, need_traffic_metrics: bool = False, use_full_traffic_metrics: bool = False, traffic_metrics_duration_s: float = 0.0, need_worker_states: bool = False, need_worker_fpm: bool = False, at_monotonic_s: Optional[float] = None) -> None

at_s is an absolute wall-clock time for the native adapter and a simulated time for replay. at_monotonic_s is the matching scheduler timestamp used to make observation-prefetch and plugin-dispatch cadence decisions against the same clock value.

components/src/dynamo/planner/core/types.py#L21

Public methods

init

1
__init__(at_s: float, run_load_scaling: bool = False, run_throughput_scaling: bool = False, need_traffic_metrics: bool = False, use_full_traffic_metrics: bool = False, traffic_metrics_duration_s: float = 0.0, need_worker_states: bool = False, need_worker_fpm: bool = False, at_monotonic_s: Optional[float] = None) -> None

No summary available.

source

Raised when a required component role is not found in the deployment.

1
from dynamo.planner.errors import SubComponentNotFoundError

1
SubComponentNotFoundError(sub_component_type: str)

This occurs when the DynamoGraphDeployment doesn’t contain any component with the requested role (e.g., ‘prefill’, ‘decode’).

components/src/dynamo/planner/errors.py#L168

Public methods

init

1
__init__(sub_component_type: str)

No summary available.

source

No summary available.

1
from dynamo.planner import SubComponentType

components/src/dynamo/planner/config/defaults.py#L123

No summary available.

1
from dynamo.planner import TargetReplica

components/src/dynamo/planner/config/defaults.py#L128

What the adapter provides to the core on each tick.

1
from dynamo.planner.core import TickInput

1
TickInput(now_s: float, traffic: Optional[TrafficObservation] = None, worker_counts: Optional[WorkerCounts] = None, fpm_observations: Optional[FpmObservations] = None) -> None

Fields are filled according to the previous ScheduledTick’s declared requirements.

components/src/dynamo/planner/core/types.py#L82

Public methods

init

1
__init__(now_s: float, traffic: Optional[TrafficObservation] = None, worker_counts: Optional[WorkerCounts] = None, fpm_observations: Optional[FpmObservations] = None) -> None

No summary available.

source

Aggregated traffic metrics over an observation window.

1
from dynamo.planner.core import TrafficObservation

1
TrafficObservation(duration_s: float, num_req: float, isl: float, osl: float, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

components/src/dynamo/planner/core/types.py#L50

Public methods

init

1
__init__(duration_s: float, num_req: float, isl: float, osl: float, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

No summary available.

source

Raised when the model name is not the same as the user provided model name

1
from dynamo.planner.errors import UserProvidedModelNameMismatchError

1
UserProvidedModelNameMismatchError(model_name: str, user_provided_model_name: str)

components/src/dynamo/planner/errors.py#L132

Public methods

init

1
__init__(model_name: str, user_provided_model_name: str)

No summary available.

source

Coordinate planner scaling decisions for non-native environments.

1
from dynamo.planner import VirtualConnector

1
VirtualConnector(runtime: DistributedRuntime, dynamo_namespace: str, worker_info_provider: WorkerInfoProvider, model_name: Optional[str] = None)

The connector does not scale a deployment directly. It publishes decisions through the Dynamo runtime’s VirtualConnectorCoordinator; the deployment environment consumes them with VirtualConnectorClient and reports scaling status back to the coordinator.

Virtual deployments do not have a Kubernetes API from which to derive worker component and endpoint metadata. They therefore require a worker_info_provider that resolves WorkerInfo from runtime MDC. The planner factory normally supplies its RuntimeFpmProvider, sharing the same runtime discovery source used for forward-pass metrics.

components/src/dynamo/planner/connectors/virtual.py#L30

Public methods

init

1
__init__(runtime: DistributedRuntime, dynamo_namespace: str, worker_info_provider: WorkerInfoProvider, model_name: Optional[str] = None)

Initialize a virtual deployment connector.

Parameters

source

get_worker_info

1
get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

No summary available.

source

async_init

1
async_init()

Async initialization that must be called after init

source

add_component

1
add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component by increasing its replica count by 1

source

remove_component

1
remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component by decreasing its replica count by 1

source

set_component_replicas

1
set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set the replicas for multiple components at once

source

validate_deployment

1
validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True)

Validate the deployment

source

wait_for_deployment_ready

1
wait_for_deployment_ready(include_planner: bool = True)

Wait for the deployment to be ready

source

get_worker_runtime_namespace

1
get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

No summary available.

source

get_actual_worker_counts

1
get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Read active workers from discovery and scaling status from the client ack.

Coordinator worker counts are desired targets, not observations. Runtime endpoint discovery is the source of truth for active workers, while the coordinator’s decision acknowledgement indicates whether scaling is still in progress.

source

get_model_name

1
get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get the model name from the deployment

source

get_gpu_counts

1
get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

Virtual deployments do not expose GPU shape through the coordinator.

source

Static per-engine capabilities discovered at startup from MDC.

1
from dynamo.planner.core import WorkerCapabilities

1
WorkerCapabilities(prefill: Optional[EngineCapabilities] = None, decode: Optional[EngineCapabilities] = None) -> None

Provided once when constructing the planner core. In native mode these come from WorkerInfo (resolved via MDC / DGD); in replay they come from the simulated engine args.

For agg mode, only decode is populated (single engine type).

components/src/dynamo/planner/core/types.py#L214

Public methods

init

1
__init__(prefill: Optional[EngineCapabilities] = None, decode: Optional[EngineCapabilities] = None) -> None

No summary available.

source

Current worker inventory as reported by the adapter.

1
from dynamo.planner.core import WorkerCounts

1
WorkerCounts(ready_num_prefill: Optional[int] = None, ready_num_decode: Optional[int] = None, expected_num_prefill: Optional[int] = None, expected_num_decode: Optional[int] = None, prefill_scaling_in_progress: bool = False, decode_scaling_in_progress: bool = False) -> None

components/src/dynamo/planner/core/types.py#L62

Public methods

init

1
__init__(ready_num_prefill: Optional[int] = None, ready_num_decode: Optional[int] = None, expected_num_prefill: Optional[int] = None, expected_num_decode: Optional[int] = None, prefill_scaling_in_progress: bool = False, decode_scaling_in_progress: bool = False) -> None