Dynamic Power Sloshing#

Summary#

DGX Station GB300 systems share a fixed total power budget of 1600 W between the GB110 module (the main GB300 GPU and TH500, referred to internally as TMP) and an optional RTX add-in card. The vsloshd daemon enforces that ceiling: the power caps for the two devices must always sum to no more than 1600 W.

In its default Dynamic mode, called power sloshing, vsloshd continuously reallocates the 1600 W budget between the two devices based on real-time power draw instead of holding each device at a fixed cap. When one device runs close to its cap and needs more power, the daemon shifts power budget to that device by lowering the other device’s cap and raising the requesting device’s cap one step at a time. The daemon prioritizes the RTX card over the GB110 module: if the RTX needs power, the daemon allocates more power to it immediately. The GB110 module can reclaim power from the RTX only when the RTX has clear surplus (an anti-oscillation guard). The daemon clamps each cap to the device’s hardware minimum and maximum policy limits, keeps the combined total at or below 1600 W, and applies reductions before increases so the budget never temporarily exceeds the limit.

In Static mode, both caps are fixed. The GB110 cap is derived from the RTX maximum in the following table, and any external change to either cap is detected and reverted once per second.

Static and Initial GB110 Allocation#

At startup, and whenever Static mode is active, the GB110 cap is based on the RTX card’s maximum power rating (calcGb110Cap):

RTX Maximum Power	GB110 Power Cap
≤ 75 W	1600 W
76 W – 300 W	1250 W
> 300 W	950 W
No RTX detected	1600 W (GB110 receives the full budget)

Sloshing Thresholds and Tunables#

These settings control when and how aggressively the algorithm moves power. Except where noted, you can set them at boot in /etc/default/vsloshd or change them live over D-Bus or in the vsloshd-dash dashboard.

Setting	D-Bus Property	Environment Variable	Default	Range
Total system budget	— (fixed)	—	1600 W	Fixed constant
Headroom threshold	`SloshingThreshold`	`VSLOSHD_THRESHOLD`	50 W	1–500 W
Step	`SloshingStep`	`VSLOSHD_STEP`	50 W	1–200 W
Cycle interval	`SloshingIntervalMilliseconds`	`VSLOSHD_INTERVAL_MS`	500 ms	≥ 1 ms (dashboard 1–60,000)
Power sensor read cadence	— (in `smbpbid`)	`SMBPBI_POWER_POLL_INTERVAL_MS`	100 ms	—
Mode	`SloshingMode`	`VSLOSHD_DYNAMIC`	Dynamic	Static or Dynamic

What each setting does:

Total system budget: The fixed ceiling. The combined RTX and GB110 caps must never exceed this value. This budget is the pool that gets redistributed.
Headroom threshold: The trigger. A device is hungry when its headroom (cap − draw) falls below this value. Sloshing then shifts budget toward that device.
Step: How much budget moves per cycle. In the steady, fully allocated state, exactly one step is shifted from the non-hungry device to the hungry device. The step also sets the surplus margin (threshold + step) that the RTX must have before the GB110 is allowed to reclaim power.
Cycle interval: How often the algorithm rereads telemetry and rebalances. A shorter interval is more responsive but generates more bus traffic.
Power sensor read cadence: How often smbpbid polls RTX power draw over I2C. This cadence determines the freshness of the draw values that each sloshing cycle reads. It is kept faster than the cycle interval, so every cycle sees recent data. The smbpbid daemon owns this setting; you cannot tune it through the vsloshd D-Bus interface or dashboard. nsmd polls GB110 module power, typically every 1–5 s.
Mode: Dynamic enables sloshing. Static fixes caps from the RTX maximum power rating and reverts external changes.

Note

RTXPowerCapWatts and GB110PowerCapWatts are read-only D-Bus properties that report the caps currently in force. Each device’s cap is also clamped to the device’s hardware [min, max] policy. For example, an RTX card with a rated 150–600 W range can use only caps within that range. The policy is read from D-Bus.

What the Algorithm Does#

Each cycle (every SloshingIntervalMilliseconds, default 500 ms), the daemon:

Checks preconditions: Both the RTX and GB110 must be connected. If either device is not connected, the cycle is skipped.
Reads live telemetry: Read the current cap and current power draw for both devices over D-Bus, with RTX data from smbpbid and GB110 data from nsmd.
Validates the data: If either cap reads 0, either draw read fails or is implausible (outside 1–3000 W), or any value exceeds 5000 W, the cycle is skipped. The algorithm never reallocates on stale or invalid telemetry.
Corrects drift: If a live cap has risen above the value vsloshd last set (for example, an external change or a service restart), the expected cap is re-enforced and reallocation is skipped for this cycle.
Computes headroom: Headroom for each device is computed as cap − draw. A device is marked hungry when its headroom is below the threshold.
Decide direction, RTX-first:
- If the RTX is hungry, shift budget to it: set the GB110 cap to 1600 − RTX_cap − step, then give the RTX the remainder (1600 − new GB110 cap).
- Otherwise, if the GB110 is hungry and the RTX has clear surplus (RTX headroom > threshold + step), shift budget the other way: set the RTX cap to 1600 − GB110_cap − step, then give the GB110 the remainder. The surplus requirement prevents the two devices from oscillating.
Clamp to policy limits: Both targets are bounded to each device’s hardware [min, max]. If a hungry device cannot gain power because the other device is already at its floor, the cycle is skipped.
Enforce the ceiling: As a final guard, if the two targets sum above 1600 W, the non-priority device is trimmed so the total fits.
Apply changes safely: A 0 W cap is never written. The reduction before the increase is written so the budget is never momentarily exceeded. If the increase then fails, the reduction is reverted so both devices end the cycle back at their starting caps. A cap-write failure puts the affected device in a backoff window, so its peer is not repeatedly disturbed.
Repeat: The cycle repeats on the next interval, so the split continuously tracks workload demand. When neither device is hungry, the caps hold steady.