Managed Host State Diagrams
This document contains the complete Finite State Machine (FSM) that illustrates the lifecycle of NICo managed hosts from discovery through ingestion through instance assignment and management.
High-Level Overview
The main flow shows the primary states and transitions between them:
Failed recovery semantics
FailureCause::BiosSetupFailed is set when BIOS setup retries are exhausted during ingestion or instance deprovisioning. The Failed handler polls is_bios_setup and auto-recovers into SetBootOrder once the BMC reports success.
DPU Discovery State Details (DpuDiscoveringState)
Shows the complete DPU discovery and configuration process:
DPU Initialization State Details (DpuInitState)
Shows DPU initialization including BFB installation:
Host Initialization State Details (HostInitState)
Shows host initialization including boot order and UEFI setup:
BOM Validation State Details (BomValidating)
Shows the BOM (Bill of Materials) validation process:
Machine Validation State Details (ValidationState)
Shows the machine validation process:
Ready State Details (Ready)
Shows what can happen in Ready state:
Instance Assignment State Details (InstanceState)
Shows the complete instance assignment and management flow:
Network Readiness Gates
During initial provisioning, WaitingForNetworkConfig waits for all required
network configuration observations to match their desired versions. For a host
with associated DPUs, it also waits for aggregate health alerts that prevent
host state changes to clear. A host without associated DPUs skips both the DPU
observation and aggregate health checks in this state.
The normal agent report gives a p0 transport failure the combination of
BgpPeeringTor and PreventAllocations. The PXE gate matches that probe ID and
classification; it does not inspect the alert target. A host Replace report
takes precedence for this check. Without one, the primary DPU Replace report
takes precedence over that DPU’s Merge reports. Refer to
DPU ToR Uplink Health
for the uplink policy and classifications.
WaitingForRebootToReady repeats the aggregate health and primary p0 checks
immediately before the normal PXE ForceRestart. This closes the interval in
which health can change after the configuration check. Hosts without managed
DPUs skip this PXE signal check, including one supplied by a host Replace
report, but still receive the aggregate health check. In
WaitingForNetworkConfig, a host Replace report can supply the matching
signal before NICo looks for a primary DPU, including on a host without managed
DPUs.
Normal success proceeds directly from WaitingForNetworkConfig to
WaitingForRebootToReady. WaitingForStorageConfig and
WaitingForExtensionServicesConfig remain only so NICo can recover hosts that
persisted those states on an older release.
The following requests bypass these readiness checks:
- Instance deletion proceeds through its required restart without waiting for network health.
- An explicit custom iPXE request proceeds through its requested restart.
A live network update uses NetworkConfigUpdate/WaitingForConfigSynced. A host
with associated DPUs waits for current observations and aggregate health alerts
that prevent host state changes. A host without associated DPUs skips both
checks. This path does not apply the primary p0 PXE gate or restart the host.
Host Reprovision State Details (HostReprovisionState)
Shows the host firmware reprovision process:
DPU Reprovision State Details (DpuReprovisionState)
Shows the DPU firmware reprovision process:
WaitingForCleanup State Details
Measuring and PostAssignedMeasuring State Details
Shows the attestation measurement process.