dynamo.planner | NVIDIA Dynamo Documentation

dynamo.planner publishes 31 classes and 0 functions. Source: components/src/dynamo/planner/__init__.py

BackendFrameworkInvalidError (class)

Raised when the backend framework does not exist.

1 from dynamo.planner.errors import BackendFrameworkInvalidError

1 BackendFrameworkInvalidError(backend_framework: str)

This occurs when the DynamoGraphDeployment contains an unsupported backend framework.

components/src/dynamo/planner/errors.py#L152

Public methods

init

1 __init__(backend_framework: str)

No summary available.

source

BackendFrameworkNotFoundError (class)

Raised when the backend framework is not supported.

1 from dynamo.planner.errors import BackendFrameworkNotFoundError

1 BackendFrameworkNotFoundError()

This occurs when the DynamoGraphDeployment contains an unsupported backend framework.

components/src/dynamo/planner/errors.py#L141

Public methods

init

1 __init__()

No summary available.

source

ComponentError (class)

Base class for component type configuration issues.

1 from dynamo.planner.errors import ComponentError

This serves as a parent class for all exceptions related to component type configuration problems in DynamoGraphDeployments.

components/src/dynamo/planner/errors.py#L96

DeploymentModelNameMismatchError (class)

Raised when the model name is not the same in the deployment

1 from dynamo.planner.errors import DeploymentModelNameMismatchError

1 DeploymentModelNameMismatchError(prefill_model_name: str, decode_model_name: str)

components/src/dynamo/planner/errors.py#L114

Public methods

init

1 __init__(prefill_model_name: str, decode_model_name: str)

No summary available.

source

DeploymentValidationError (class)

Raised when deployment validation fails for multiple components.

1 from dynamo.planner.errors import DeploymentValidationError

1 DeploymentValidationError(errors: List[str])

This is used to aggregate multiple validation errors into a single exception, providing a comprehensive view of all validation issues.

components/src/dynamo/planner/errors.py#L211

Public methods

init

1 __init__(errors: List[str])

No summary available.

source

DuplicateSubComponentError (class)

Raised when multiple components have the same planner role.

1 from dynamo.planner.errors import DuplicateSubComponentError

1 DuplicateSubComponentError(sub_component_type: str, service_names: List[str])

This occurs when the DynamoGraphDeployment contains more than one component with the same role, which violates the expected uniqueness constraint.

components/src/dynamo/planner/errors.py#L188

Public methods

init

1 __init__(sub_component_type: str, service_names: List[str])

No summary available.

source

DynamoGraphDeploymentNotFoundError (class)

Raised when Parent DynamoGraphDeployment cannot be found.

1 from dynamo.planner.errors import DynamoGraphDeploymentNotFoundError

1 DynamoGraphDeploymentNotFoundError(deployment_name: str, namespace: str)

This typically occurs when:

The DYN_PARENT_DGD_K8S_NAME environment variable is not set
The referenced DynamoGraphDeployment doesn’t exist in the namespace

components/src/dynamo/planner/errors.py#L52

Public methods

init

1 __init__(deployment_name: str, namespace: str)

No summary available.

source

DynamoGraphDeploymentNotReadyError (class)

Raised when a DynamoGraphDeployment is not ready for scaling.

1 from dynamo.planner.errors import DynamoGraphDeploymentNotReadyError

1 DynamoGraphDeploymentNotReadyError(deployment_name: str, namespace: str | None = None)

components/src/dynamo/planner/errors.py#L72

Public methods

init

1 __init__(deployment_name: str, namespace: str | None = None)

No summary available.

source

EmptyTargetReplicasError (class)

Raised when target_replicas is empty or invalid.

1 from dynamo.planner.errors import EmptyTargetReplicasError

1 EmptyTargetReplicasError()

This occurs when attempting to set component replicas with an empty or invalid target_replicas dictionary.

components/src/dynamo/planner/errors.py#L224

Public methods

init

1 __init__()

No summary available.

source

EngineCapabilities (class)

Static capabilities for a single engine stage (prefill or decode).

1 from dynamo.planner.core import EngineCapabilities

1 EngineCapabilities(num_gpu: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, max_num_seqs: Optional[int] = None, context_length: Optional[int] = None, max_kv_tokens: Optional[int] = None, kv_cache_block_size: Optional[int] = None, speculative_nextn: Optional[int] = None) -> None

components/src/dynamo/planner/core/types.py#L197

Public methods

init

1 __init__(num_gpu: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, max_num_seqs: Optional[int] = None, context_length: Optional[int] = None, max_kv_tokens: Optional[int] = None, kv_cache_block_size: Optional[int] = None, speculative_nextn: Optional[int] = None) -> None

No summary available.

source

FpmObservations (class)

Per-engine ForwardPassMetrics keyed by (worker_id, dp_rank).

1 from dynamo.planner.core import FpmObservations

1 FpmObservations(prefill: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None, decode: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None) -> None

components/src/dynamo/planner/core/types.py#L74

Public methods

init

1 __init__(prefill: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None, decode: Optional[dict[tuple[str, int], ForwardPassMetrics]] = None) -> None

No summary available.

source

GlobalPlannerConnector (class)

Connector that delegates scaling decisions to a centralized GlobalPlanner.

1 from dynamo.planner import GlobalPlannerConnector

1 GlobalPlannerConnector(runtime: DistributedRuntime, dynamo_namespace: str, global_planner_namespace: str, global_planner_component: str = 'GlobalPlanner', model_name: Optional[str] = None)

This connector wraps RemotePlannerClient and implements the InfraScaler interface, allowing planner_core.py to treat global-planner environment mode consistently with kubernetes and virtual modes.

components/src/dynamo/planner/connectors/global_planner.py#L37

Public methods

init

1 __init__(runtime: DistributedRuntime, dynamo_namespace: str, global_planner_namespace: str, global_planner_component: str = 'GlobalPlanner', model_name: Optional[str] = None)

Initialize GlobalPlannerConnector.

Parameters

runtime

DistributedRuntime

Distributed runtime for communication

dynamo_namespace

str

Local dynamo namespace (caller identification)

global_planner_namespace

str

Namespace where GlobalPlanner is deployed

global_planner_component

str

Component name of GlobalPlanner (default: “GlobalPlanner”)

model_name

Optional[str]

Optional model name (will be managed remotely if not provided)

source

async_init

1 async_init()

Async initialization - creates RemotePlannerClient

source

set_predicted_load

1 set_predicted_load(num_requests: Optional[float], isl: Optional[float], osl: Optional[float])

Set predicted load for inclusion in next scale request.

This is called by planner_core.py before calling set_component_replicas.

source

set_component_replicas

1 set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set component replicas by delegating to GlobalPlanner.

Sends a ScaleRequest to the GlobalPlanner with the target replica configurations.

Parameters

target_replicas

list[TargetReplica]

List of target replica configurations

blocking

bool

Whether to wait for scaling completion (passed to GlobalPlanner)

Raises

EmptyTargetReplicasError — If target_replicas is empty
RuntimeError — If remote_client is not initialized or the response indicates a hard error (e.g., authorization denied, K8s exception). A REJECTED response is NOT raised — it is logged as a warning and treated as a no-op for this tick.

source

add_component

1 add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component (not supported for GlobalPlanner).

GlobalPlanner only supports batch operations via set_component_replicas.

source

remove_component

1 remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component (not supported for GlobalPlanner).

GlobalPlanner only supports batch operations via set_component_replicas.

source

validate_deployment

1 validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True) -> None

Validate deployment (no-op for GlobalPlanner).

The GlobalPlanner validates the deployment on its side, so local validation is not needed in delegating mode.

source

wait_for_deployment_ready

1 wait_for_deployment_ready(include_planner: bool = True)

Wait for the pool’s own workers to be ready.

Even though GlobalPlanner handles cluster-wide orchestration, the pool Planner still reads its own workers’ DynamoWorkerMetadata CRs for capability discovery (get_worker_info). Without a local wait, _async_init runs within milliseconds of pod entry — long before workers register MDC — so get_worker_info falls back to defaults with context_length / max_kv_tokens unset and load-scaling silently disables itself for the pod’s lifetime.

Mirror the standalone path by delegating to the pool-local KubernetesConnector. If no local connector is available (e.g. running outside a cluster), fall back to the previous no-op so out-of-cluster callers are not blocked.

source

get_actual_worker_counts

1 get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Read ready replica counts and rollout stability from the pool’s own DGD.

GlobalPlanner orchestrates scaling, but the pool Planner pod has direct access to its own DGD status. Mirror KubernetesConnector by delegating to the pool-local connector so _scaling_in_progress observes real rollouts instead of always seeing is_stable=True.

Returns (0, 0, True) when no local KubernetesConnector is available (e.g. running out-of-cluster), matching the existing capability-discovery fallback path so out-of-cluster callers aren’t blocked.

source

get_worker_runtime_namespace

1 get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

Resolve the pool-local worker runtime namespace when available.

source

get_gpu_counts

1 get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

Resolve pool-local GPU shape when available.

source

get_worker_info

1 get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

Resolve per-worker capabilities from the pool’s own MDC/DGD.

Without this, resolve_worker_info falls through to build_worker_info_from_defaults which leaves context_length and max_kv_tokens unset, and load_scaling’s easy-mode decisions bail out every tick — so the pool Planner silently sends no ScaleRequests.

source

get_model_name

1 get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get model name.

Prefers the value provided at init time, then the pool’s own DGD container args (via the local KubernetesConnector), and finally falls back to a placeholder indicating the model is managed remotely.

source

KubernetesConnector (class)

No summary available.

1 from dynamo.planner import KubernetesConnector

1 KubernetesConnector(dynamo_namespace: str, model_name: Optional[str] = None, k8s_namespace: Optional[str] = None, parent_dgd_name: Optional[str] = None, raise_not_ready: bool = False)

components/src/dynamo/planner/connectors/kubernetes.py#L63

Public methods

init

1 __init__(dynamo_namespace: str, model_name: Optional[str] = None, k8s_namespace: Optional[str] = None, parent_dgd_name: Optional[str] = None, raise_not_ready: bool = False)

No summary available.

source

async_init

1 async_init()

No-op asynchronous lifecycle hook.

source

get_worker_runtime_namespace

1 get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

Return the Dynamo namespace used by the current worker generation.

Newer operators publish the effective runtime namespace on the worker component status. Older operators expose only the active worker hash, so the planner falls back to appending that hash only for Deployment-backed and LeaderWorkerSet-backed workers.

source

add_component

1 add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component by increasing its replica count by 1

source

remove_component

1 remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component by decreasing its replica count by 1

source

validate_deployment

1 validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True)

Verify that the deployment contains prefill/decode components and the model name exists. Allows explicit component-name overrides when the caller provides them.

Raises

DynamoGraphDeploymentNotFoundError — If the deployment is not found
DeploymentValidationError — If the deployment does not contain required prefill/decode components

source

get_model_name

1 get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get the model name from the current deployment.

source

get_gpu_counts

1 get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[int, int]

Get the GPU counts for prefill and decode components.

source

get_frontend_metrics_url

1 get_frontend_metrics_url(port: int = 8000) -> Optional[str]

Auto-discover the frontend component’s metrics URL from the DGD.

Iterates DGD components to find the component with type “frontend”, then constructs the in-cluster URL using the operator’s naming convention: http://{dgd_name}-{component_name_lowercase}:{port}/metrics

Returns

Optional[str] — The metrics URL string, or None if no frontend component is found.

source

wait_for_deployment_ready

1 wait_for_deployment_ready(include_planner: bool = True)

Wait for the deployment to be ready.

Parameters

include_planner

bool

If False, skip the planner component when checking readiness. This lets the planner read MDC from worker pods without waiting for itself to be marked ready in the DGD.

source

get_worker_info

1 get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

Get WorkerInfo for a sub-component, trying MDC first, then fallbacks.

Parameters

sub_component_type

SubComponentType

PREFILL or DECODE

backend

str

Backend framework name (for default fallback)

source

get_actual_worker_counts

1 get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Get actual ready worker counts for prefill and decode from DGD status.

Returns

int — tuple[int, int, bool]: (prefill_count, decode_count, is_stable)
int — - is_stable: False if any component is in a rollout (scaling should be skipped)

source

set_component_replicas

1 set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set the replicas for multiple components at once

source

ModelNameNotFoundError (class)

Raised when the model name is not found in the deployment

1 from dynamo.planner.errors import ModelNameNotFoundError

1 ModelNameNotFoundError()

components/src/dynamo/planner/errors.py#L106

Public methods

init

1 __init__()

No summary available.

source

PlannerConnector (class)

No summary available.

1 from dynamo.planner import PlannerConnector

components/src/dynamo/planner/connectors/base.py#L28

Public methods

async_init

1 async_init() -> None

No summary available.

source

validate_deployment

1 validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True) -> None

No summary available.

source

wait_for_deployment_ready

1 wait_for_deployment_ready(include_planner: bool = True) -> None

No summary available.

source

get_model_name

1 get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

No summary available.

source

get_gpu_counts

1 get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

No summary available.

source

get_actual_worker_counts

1 get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

No summary available.

source

set_component_replicas

1 set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True) -> None

No summary available.

source

PlannerEffects (class)

What the core returns after processing a tick.

1 from dynamo.planner.core import PlannerEffects

1 PlannerEffects(scale_to: Optional[ScalingDecision] = None, next_tick: Optional[ScheduledTick] = None, diagnostics: TickDiagnostics = TickDiagnostics()) -> None

components/src/dynamo/planner/core/types.py#L188

Public methods

init

1 __init__(scale_to: Optional[ScalingDecision] = None, next_tick: Optional[ScheduledTick] = None, diagnostics: TickDiagnostics = TickDiagnostics()) -> None

No summary available.

source

PlannerError (class)

Base exception for all planner-related errors.

1 from dynamo.planner.errors import PlannerError

This serves as the root exception class for all custom exceptions in the planner module, allowing for broad exception catching when needed.

components/src/dynamo/planner/errors.py#L42

PlannerScalingState (class)

Shared in-memory scaling state for all planner modes.

1 from dynamo.planner.core import PlannerScalingState

1 PlannerScalingState(config: PlannerConfig, capabilities: Optional[WorkerCapabilities] = None) -> None

Owns perf models, throughput lower bounds, worker inventory, last-value runtime metadata, and all scaling decision logic. It deliberately has no runtime dependencies. Load prediction state lives in the builtin PREDICT plugin and is passed in explicitly.

Builtin orchestrator plugins use this class directly as their private shared core while the remaining cross-plugin state is being split into explicit pipeline artifacts.

components/src/dynamo/planner/core/state_machine.py#L48

Public methods

init

1 __init__(config: PlannerConfig, capabilities: Optional[WorkerCapabilities] = None) -> None

No summary available.

source

update_capabilities

1 update_capabilities(capabilities: WorkerCapabilities) -> None

Replace the current worker capabilities.

source

load_benchmark_fpms

1 load_benchmark_fpms(prefill_fpms: Optional[list[ForwardPassMetrics]] = None, decode_fpms: Optional[list[ForwardPassMetrics]] = None, agg_fpms: Optional[list[ForwardPassMetrics]] = None) -> None

No summary available.

source

begin_tick

1 begin_tick() -> None

Reset per-tick diagnostics before builtin plugins run.

source

observe_worker_counts

1 observe_worker_counts(counts: WorkerCounts) -> None

No summary available.

source

observe_fpm

1 observe_fpm(obs: FpmObservations) -> None

No summary available.

source

observe_runtime_metadata

1 observe_runtime_metadata(*, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

Update last-value runtime metadata without touching prediction history.

source

install_regressions

1 install_regressions(*, prefill: Optional[PlannerEnginePerfModel] = None, decode: Optional[PlannerEnginePerfModel] = None, agg: Optional[PlannerEnginePerfModel] = None) -> None

No summary available.

source

advance_load

1 advance_load(obs: FpmObservations, *, predicted_kv_hit_rate: Optional[float] = None, predicted_accept_length: Optional[float] = None) -> Optional[ScalingDecision]

No summary available.

source

advance_throughput_from_prediction

1 advance_throughput_from_prediction(traffic: TrafficObservation, *, predicted_num_req: Optional[float], predicted_isl: Optional[float], predicted_osl: Optional[float], predicted_kv_hit_rate: Optional[float], predicted_accept_length: Optional[float] = None) -> Optional[ScalingDecision]

Run the throughput decision using PREDICT-stage output.

The PREDICT plugin owns load prediction history. This method consumes only explicit prediction output so PROPOSE never re-runs prediction or depends on hidden predictor state.

source

diagnostics

1 diagnostics() -> TickDiagnostics

No summary available.

source

SLAPlannerDefaults (class)

No summary available.

1 from dynamo.planner import SLAPlannerDefaults

components/src/dynamo/planner/config/defaults.py#L44

ScalingDecision (class)

Desired replica counts. None means the core has no opinion on that component (e.g. prefill-only planner leaves decode as None).

1 from dynamo.planner.core import ScalingDecision

1 ScalingDecision(num_prefill: Optional[int] = None, num_decode: Optional[int] = None) -> None

components/src/dynamo/planner/core/types.py#L96

Public methods

init

1 __init__(num_prefill: Optional[int] = None, num_decode: Optional[int] = None) -> None

No summary available.

source

ScheduledTick (class)

Declares when the core next needs to be called, what data it needs, and what decisions to make.

1 from dynamo.planner.core import ScheduledTick

1 ScheduledTick(at_s: float, run_load_scaling: bool = False, run_throughput_scaling: bool = False, need_traffic_metrics: bool = False, use_full_traffic_metrics: bool = False, traffic_metrics_duration_s: float = 0.0, need_worker_states: bool = False, need_worker_fpm: bool = False, at_monotonic_s: Optional[float] = None) -> None

at_s is an absolute wall-clock time for the native adapter and a simulated time for replay. at_monotonic_s is the matching scheduler timestamp used to make observation-prefetch and plugin-dispatch cadence decisions against the same clock value.

components/src/dynamo/planner/core/types.py#L21

Public methods

init

1 __init__(at_s: float, run_load_scaling: bool = False, run_throughput_scaling: bool = False, need_traffic_metrics: bool = False, use_full_traffic_metrics: bool = False, traffic_metrics_duration_s: float = 0.0, need_worker_states: bool = False, need_worker_fpm: bool = False, at_monotonic_s: Optional[float] = None) -> None

No summary available.

source

SubComponentNotFoundError (class)

Raised when a required component role is not found in the deployment.

1 from dynamo.planner.errors import SubComponentNotFoundError

1 SubComponentNotFoundError(sub_component_type: str)

This occurs when the DynamoGraphDeployment doesn’t contain any component with the requested role (e.g., ‘prefill’, ‘decode’).

components/src/dynamo/planner/errors.py#L165

Public methods

init

1 __init__(sub_component_type: str)

No summary available.

source

SubComponentType (class)

No summary available.

1 from dynamo.planner import SubComponentType

components/src/dynamo/planner/config/defaults.py#L106

TargetReplica (class)

No summary available.

1 from dynamo.planner import TargetReplica

components/src/dynamo/planner/config/defaults.py#L111

TickInput (class)

What the adapter provides to the core on each tick.

1 from dynamo.planner.core import TickInput

1 TickInput(now_s: float, traffic: Optional[TrafficObservation] = None, worker_counts: Optional[WorkerCounts] = None, fpm_observations: Optional[FpmObservations] = None) -> None

Fields are filled according to the previous ScheduledTick’s declared requirements.

components/src/dynamo/planner/core/types.py#L82

Public methods

init

1 __init__(now_s: float, traffic: Optional[TrafficObservation] = None, worker_counts: Optional[WorkerCounts] = None, fpm_observations: Optional[FpmObservations] = None) -> None

No summary available.

source

TrafficObservation (class)

Aggregated traffic metrics over an observation window.

1 from dynamo.planner.core import TrafficObservation

1 TrafficObservation(duration_s: float, num_req: float, isl: float, osl: float, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

components/src/dynamo/planner/core/types.py#L50

Public methods

init

1 __init__(duration_s: float, num_req: float, isl: float, osl: float, kv_hit_rate: Optional[float] = None, accept_length: Optional[float] = None) -> None

No summary available.

source

UserProvidedModelNameMismatchError (class)

Raised when the model name is not the same as the user provided model name

1 from dynamo.planner.errors import UserProvidedModelNameMismatchError

1 UserProvidedModelNameMismatchError(model_name: str, user_provided_model_name: str)

components/src/dynamo/planner/errors.py#L129

Public methods

init

1 __init__(model_name: str, user_provided_model_name: str)

No summary available.

source

VirtualConnector (class)

Coordinate planner scaling decisions for non-native environments.

1 from dynamo.planner import VirtualConnector

1 VirtualConnector(runtime: DistributedRuntime, dynamo_namespace: str, worker_info_provider: WorkerInfoProvider, model_name: Optional[str] = None)

The connector does not scale a deployment directly. It publishes decisions through the Dynamo runtime’s VirtualConnectorCoordinator; the deployment environment consumes them with VirtualConnectorClient and reports scaling status back to the coordinator.

Virtual deployments do not have a Kubernetes API from which to derive worker component and endpoint metadata. They therefore require a worker_info_provider that resolves WorkerInfo from runtime MDC. The planner factory normally supplies its RuntimeFpmProvider, sharing the same runtime discovery source used for forward-pass metrics.

components/src/dynamo/planner/connectors/virtual.py#L30

Public methods

init

1 __init__(runtime: DistributedRuntime, dynamo_namespace: str, worker_info_provider: WorkerInfoProvider, model_name: Optional[str] = None)

Initialize a virtual deployment connector.

Parameters

runtime

DistributedRuntime

Distributed runtime used for coordination and discovery.

dynamo_namespace

str

Namespace containing the virtual deployment.

worker_info_provider

WorkerInfoProvider

Required source of runtime WorkerInfo/MDC used to locate worker endpoints. construct_environment normally provides a RuntimeFpmProvider.

model_name

Optional[str]

Model name reported by the deployment.

source

get_worker_info

1 get_worker_info(sub_component_type: SubComponentType, backend: str = 'vllm') -> WorkerInfo

No summary available.

source

async_init

1 async_init()

Async initialization that must be called after init

source

add_component

1 add_component(sub_component_type: SubComponentType, blocking: bool = True)

Add a component by increasing its replica count by 1

source

remove_component

1 remove_component(sub_component_type: SubComponentType, blocking: bool = True)

Remove a component by decreasing its replica count by 1

source

set_component_replicas

1 set_component_replicas(target_replicas: list[TargetReplica], blocking: bool = True)

Set the replicas for multiple components at once

source

validate_deployment

1 validate_deployment(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None, require_prefill: bool = True, require_decode: bool = True)

Validate the deployment

source

wait_for_deployment_ready

1 wait_for_deployment_ready(include_planner: bool = True)

Wait for the deployment to be ready

source

get_worker_runtime_namespace

1 get_worker_runtime_namespace(base_dynamo_namespace: str) -> str

No summary available.

source

get_actual_worker_counts

1 get_actual_worker_counts(prefill_component_name: Optional[str] = None, decode_component_name: Optional[str] = None) -> tuple[int, int, bool]

Read active workers from discovery and scaling status from the client ack.

Coordinator worker counts are desired targets, not observations. Runtime endpoint discovery is the source of truth for active workers, while the coordinator’s decision acknowledgement indicates whether scaling is still in progress.

source

get_model_name

1 get_model_name(require_prefill: bool = True, require_decode: bool = True) -> str

Get the model name from the deployment

source

get_gpu_counts

1 get_gpu_counts(require_prefill: bool = True, require_decode: bool = True) -> tuple[Optional[int], Optional[int]]

Virtual deployments do not expose GPU shape through the coordinator.

source

WorkerCapabilities (class)

Static per-engine capabilities discovered at startup from MDC.

1 from dynamo.planner.core import WorkerCapabilities

1 WorkerCapabilities(prefill: Optional[EngineCapabilities] = None, decode: Optional[EngineCapabilities] = None) -> None

Provided once when constructing the planner core. In native mode these come from WorkerInfo (resolved via MDC / DGD); in replay they come from the simulated engine args.

For agg mode, only decode is populated (single engine type).

components/src/dynamo/planner/core/types.py#L210

Public methods

init

1 __init__(prefill: Optional[EngineCapabilities] = None, decode: Optional[EngineCapabilities] = None) -> None

No summary available.

source

WorkerCounts (class)

Current worker inventory as reported by the adapter.

1 from dynamo.planner.core import WorkerCounts

1 WorkerCounts(ready_num_prefill: Optional[int] = None, ready_num_decode: Optional[int] = None, expected_num_prefill: Optional[int] = None, expected_num_decode: Optional[int] = None, prefill_scaling_in_progress: bool = False, decode_scaling_in_progress: bool = False) -> None

components/src/dynamo/planner/core/types.py#L62

Public methods

init

1 __init__(ready_num_prefill: Optional[int] = None, ready_num_decode: Optional[int] = None, expected_num_prefill: Optional[int] = None, expected_num_decode: Optional[int] = None, prefill_scaling_in_progress: bool = False, decode_scaling_in_progress: bool = False) -> None

No summary available.

source

WorkerInfo (class)

Consolidated worker metadata for the planner.

1 from dynamo.planner import WorkerInfo

1 WorkerInfo(k8s_name: Optional[str] = None, component_name: Optional[str] = None, endpoint: Optional[str] = None, model_name: Optional[str] = None, total_kv_blocks: Optional[int] = None, kv_cache_block_size: Optional[int] = None, max_num_seqs: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, context_length: Optional[int] = None, speculative_nextn: Optional[int] = None) -> None

Populated from MDC (DynamoWorkerMetadata CRs) in Kubernetes mode, with fallback to DGD container-arg parsing, then hard-coded defaults.

components/src/dynamo/planner/monitoring/worker_info.py#L26

Public methods

summary

1 summary() -> str

No summary available.

source

init

1 __init__(k8s_name: Optional[str] = None, component_name: Optional[str] = None, endpoint: Optional[str] = None, model_name: Optional[str] = None, total_kv_blocks: Optional[int] = None, kv_cache_block_size: Optional[int] = None, max_num_seqs: Optional[int] = None, max_num_batched_tokens: Optional[int] = None, context_length: Optional[int] = None, speculative_nextn: Optional[int] = None) -> None

No summary available.

source