Skip to navigation
Manage SandboxesOperate Sandboxes

Update Sandboxes

View as Markdown

Update the host CLI first, then check whether existing sandboxes need rebuilds. The standard installer follows the admin-promoted lkg release tag by default.

Understand Agent Version Pins

NemoClaw pins the selected agent version into the sandbox image at build time rather than updating it at runtime. Existing sandboxes do not auto-upgrade when a newer NemoClaw release ships a newer agent version. Run nemohermes <name> status to see the agent version currently running in a sandbox. After updating the host CLI, run nemohermes <name> rebuild to recreate the sandbox with the newer pin while transferring the complete OpenShell-provided native home and workspace, apart from the documented credential and control-plane exclusions.

The Hermes agent manifest expects Hermes 0.21.3 from release tag v2026.9.14 and declares the hermes --version probe command.

Use the Automatic Upgrade Path

Run the maintained installer to update the host CLI and reconcile registered sandboxes.

The maintained OpenShell 0.0.116 pin limits a routed sandbox name to 19 characters and rejects consecutive hyphens. A NemoClaw sandbox name must contain 1 to 19 characters. It must start with a lowercase letter, use lowercase letters, numbers, and single internal hyphens, and end with a letter or number. Before the OpenShell upgrade, the installer checks registered sandboxes for the selected gateway. If the installer finds incompatible names, it lists each name and exits before it can:

  • Prepare the NemoClaw CLI for the upgrade.
  • Start a backup of the registered sandboxes.
  • Retire the selected OpenShell gateway.
  • Install OpenShell 0.0.116.
  • Recreate a sandbox.

NemoClaw does not automatically truncate or change the sandbox name. Use the installed NemoClaw and OpenShell versions to create a replacement sandbox with a compatible name. Then use Transfer State Manually to move the required state into the replacement sandbox. After you verify the transferred state, destroy the incompatible sandbox with the installed NemoClaw and OpenShell versions. Then rerun the installer. If you manually retired the gateway, restore the previously installed OpenShell version and its gateway before you transfer the sandbox state.

curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash
nemohermes upgrade-sandboxes --check

The default installer preserves a newer installed NemoClaw release. When a NemoClaw CLI is installed, it also stops without changing the CLI when it cannot verify the maintained lkg release. To intentionally install an older release from a fixed source, select its version tag or full commit SHA with NEMOCLAW_INSTALL_TAG.

curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=vX.Y.Z bash

If a support workflow asks you to pass the maintained tag explicitly, clear any inherited NEMOCLAW_INSTALL_REF and set NEMOCLAW_INSTALL_TAG on the bash side of the install pipeline. The same downgrade protection applies to an explicit lkg selection.

curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=lkg bash

During the automatic upgrade path, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox. If any sandbox is skipped or fails, the installer exits before it changes the gateway. For a failed required pre-upgrade backup, NemoClaw removes the incomplete timestamped backup directory so it cannot appear as a recovery point. If that removal fails, the installer reports the retained path; preserve it and do not use it for recovery. After backup, the installer requires any existing OpenShell executable it will use to report a version, then compares that version with the supported range declared by the prepared current source. Before it retires an out-of-range gateway, the installer retires every applicable registered legacy forward through its exact sandbox, bind, port, process, gateway, and workspace identity, including Hermes dashboard, active messaging, and declared agent forwards. If it cannot prove that retirement, it exits before retiring the gateway and preserves every completed backup. It retires the running gateway before replacing an out-of-range OpenShell installation, keeps the gateway when the installed version is supported, and stops without retiring it when the installed version or supported range cannot be validated. If OpenShell lifecycle commands cannot retire the gateway, the installer can use verified service or PID-file evidence on Linux and macOS. For the default gateway on port 8080, Linux first checks verified systemd service evidence for nemoclaw-openshell-gateway.service, while macOS first checks verified Homebrew gateway service evidence. On either platform, the installer can then use a verified NemoClaw-managed gateway PID file for the configured gateway port. If macOS cannot obtain a conclusive lsof listener observation for a stale PID, the installer stops retirement and preserves the PID file, OpenShell registration, and every sandbox backup. After the service or PID-file check confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration. If registration removal fails, the installer warns that onboarding will replace the stale registration. If neither fallback can verify and stop the process, the installer stops after backup, preserves every sandbox backup, and reports the failed operation. After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before sandbox recovery. This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement. After the installer replaces the packaged gateway, it waits up to 30 seconds for the gateway port to bind before sandbox recovery begins. If the gateway does not bind, the installer stops and preserves the backups. If the forced installation fails, the installer does not stage a gateway service or start sandbox recovery, preserves the backups, and reports how to retry.

Resume a Manually Prepared Upgrade

Use this path only with the current release CLI. If the installer has not already reported that it prepared the current CLI, start the supported automatic flow first:

curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_ACCEPT_EXPERIMENTAL_OPENSHELL_UPGRADE=1 bash

That flow prepares the current CLI before it replaces OpenShell. If it completes, do not run the manual steps below. Continue manually only when the automatic flow stops after preparing the CLI or a support workflow directs you to this recovery path. Set the prepared-upgrade state only after strict backup succeeds, every applicable registered legacy forward is retired, and the gateway is retired. Strict backup can report a confirmed stranded registry sandbox without creating a backup for it because both the selected gateway and the container provider prove that no sandbox runtime remains. That record requires the cleanup or recreation action printed by nemohermes backup-all, but it does not block a prepared upgrade after the strict command succeeds. Run the backup and forward-retirement step with the current prepared NemoClaw CLI before you retire the gateway:

selected_port=8080
gateway_name=nemoclaw
NEMOCLAW_GATEWAY_PORT="$selected_port" NEMOCLAW_REQUIRE_ALL_SANDBOX_BACKUPS=1 nemohermes backup-all --retire-legacy-forwards && \
  openshell gateway destroy -g "$gateway_name"

For a non-default gateway, replace 8080 with its port and set gateway_name to nemoclaw-<selected-port>. If an older OpenShell release rejects the named destroy command for the default nemoclaw gateway, run openshell gateway destroy instead. Do not substitute openshell gateway remove: removal can unregister a gateway without stopping its process. Set NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 only after the command pipeline exits successfully. Confirm that the strict backup summary reports 0 failed, 0 skipped and that the Legacy dashboard forwards: summary appears before gateway destruction. A confirmed stranded record has no backup to reuse; complete the reported cleanup or recreation action during recovery. If either command fails or cannot prove a retirement, resolve that failure while the completed backups remain intact. Set NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 on the bash side of the install pipeline when you rerun the installer.

curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_GATEWAY_PORT="$selected_port" NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 bash

Set NEMOCLAW_GATEWAY_PORT=8080 when you prepared the default gateway. Omitting it selects port 8080 only when no automatic alternate-port marker exists, so an explicit value prevents a previously recorded alternate environment from receiving the prepared upgrade. NEMOCLAW_OPENSHELL_UPGRADE_PREPARED asserts that you completed those steps, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range. It reuses the latest backups, forces installation of the OpenShell version pinned by the prepared source, and starts recovery only after that installation succeeds. If the installation fails, rerun the same install-pipeline command to preserve NEMOCLAW_GATEWAY_PORT and NEMOCLAW_OPENSHELL_UPGRADE_PREPARED. Outside these forced paths, an in-range source checkout preserves an existing OpenShell installation and installs OpenShell only when it is missing.

Reconcile Registered Sandboxes

After the host CLI and OpenShell update, the installer runs nemohermes upgrade-sandboxes --auto to reconcile the existing sandboxes.

During installer-driven recovery, every sandbox selected for recovery requires a validated latest backup. The backup’s sandbox and agent identities must match the registry. The registry must contain positive evidence that NemoClaw managed the sandbox image, unless the name pre-fingerprint confirmation below applies. If the replacement gateway reports a stale sandbox as Ready or Running, the installer reuses the validated pre-upgrade backup. It does not attempt another backup from the replaced legacy runtime.

For a listed pre-fingerprint OpenClaw or Hermes registry entry, the installer’s explicit managed-image confirmation is the only exception to the registry evidence requirement. In a non-interactive run, set NEMOCLAW_CONFIRM_LEGACY_MANAGED_RECREATE to a JSON array that equals the normalized array of names printed by the installer, such as ["my-assistant","preserve-hermes"], only after you verify every named sandbox used a managed image.

Legacy managed-image confirmation never overrides recorded custom-image evidence.

The installer attempts every eligible recovery and exits with a nonzero status if any recovery fails. Recovery preserves each sandbox’s recorded GPU selection and runs before generic host admission. When recovery rebuilds an intentionally stopped sandbox, NemoClaw returns the replacement to Stopped. Recovery fails if OpenShell cannot confirm that phase or a required post-stop reconciliation fails. Rerun the installer to reconcile the stopped state without rebuilding a current replacement again. Fresh onboarding and required DGX Station reconciliation still require host admission.

Interpret Automatic Recovery Results

Use the installer’s completion message to distinguish these outcomes:

  • Confirmed recovery: The installer reports that existing sandboxes were recovered and upgraded. It skips generic onboarding when no Station reconciliation remains necessary.
  • Unconfirmed recovery output: The recovery command succeeded, but the installer could not inspect its output. The installer reports Installation completed with warnings and skips generic onboarding. Inspect the output of the read-only nemohermes upgrade-sandboxes --check command before further action. A successful recovery command alone does not confirm that every sandbox recovered.
  • Sandbox not found on its recorded gateway: The recovery pass exits 0 but reports the sandbox as not found rather than recovered. The installer reports Installation completed with warnings and skips generic onboarding. A prior uninstall could have removed the gateway and Docker image while preserving sandboxes.json.

If you choose to remove a stranded sandbox, follow the cleanup sequence below.

Destroying a sandbox can remove its persistent state. Preserve any required backups before cleanup. A local-only forced cleanup does not confirm that the OpenShell sandbox was deleted.

  1. Check the recorded gateway with nemohermes <name> status. This command can attempt recovery of the recorded gateway.
  2. Retry ordinary deletion with nemohermes <name> destroy.
  3. If the gateway remains unavailable, review the destroy --force fallback limits before using nemohermes <name> destroy --force. When gateway deletion is unconfirmed, this fallback removes only local state; the OpenShell sandbox can still exist. Managed MCP ownership or a sandbox-deletion timeout can prevent this fallback.
  4. If the gateway returns, verify or remove any remaining OpenShell sandbox before running nemohermes onboard. Do not infer gateway-side deletion from removal of the local record.

Use the Manual Update Flow

Keep an independent host copy before a manual update when you need an extra recovery point.

nemohermes update --yes
nemohermes upgrade-sandboxes --check

Use nemohermes upgrade-sandboxes --check to read sandbox state without starting, recovering, or selecting a gateway. The command exits nonzero when it reports stale state, an unknown version, a backup recovery condition, or a sandbox missing from its recorded gateway. Inspect each diagnostic before you select a remediation or automate the next action. When all your registered sandboxes resolve to one recorded gateway, the command queries that gateway instead of the gateway for your current NEMOCLAW_GATEWAY_PORT.

Each rebuild destroys the old container and creates a new one while transferring the complete OpenShell-provided native home/workspace. upgrade-sandboxes --check can report a sandbox as stale because the running agent version is behind, because the managed NemoClaw image fingerprint differs from the current CLI, or both.

Sandboxes created with --from-image remain pinned to their publisher-managed image digest. upgrade-sandboxes lists these sandboxes but excludes them from agent version and image drift classification, including with --auto. After you update NemoClaw, run nemohermes <sandbox-name> rebuild to revalidate the external image and recreate the sandbox from its recorded digest.

Custom-image sandboxes created with --from <Dockerfile> are not marked stale solely by image fingerprint, so an upgrade check does not accidentally replace them with the default image. Paths outside the native home/workspace, such as packages installed into the container image’s system directories, are not preserved. Packages and other files installed under the native home are preserved.