Manage SandboxesOperate Sandboxes

Update Sandboxes

View as Markdown

Update the host CLI first, then check whether existing sandboxes need rebuilds. The standard installer follows the admin-promoted lkg release tag by default.

Understand Agent Version Pins

NemoClaw pins the selected agent version into the sandbox image at build time rather than updating it at runtime. Existing sandboxes do not auto-upgrade when a newer NemoClaw release ships a newer agent version. Run nemo-deepagents <name> status to see the agent version currently running in a sandbox. After updating the host CLI, run nemo-deepagents <name> rebuild to recreate the sandbox with the newer pin while preserving the agent state that NemoClaw manages.

The langchain-deepagents-code agent manifest declares the expected Deep Agents Code version and the dcode --version probe command.

Use the Automatic Upgrade Path

Run the maintained installer to update the host CLI and reconcile registered sandboxes.

The maintained OpenShell 0.0.106 pin limits a routed sandbox name to 19 characters and rejects consecutive hyphens. A NemoClaw sandbox name must contain 1 to 19 characters. It must start with a lowercase letter, use lowercase letters, numbers, and single internal hyphens, and end with a letter or number. Before the OpenShell upgrade, the installer checks registered sandboxes for the selected gateway. If the installer finds incompatible names, it lists each name and exits before it can:

  • Prepare the NemoClaw CLI for the upgrade.
  • Start a backup of the registered sandboxes.
  • Retire the selected OpenShell gateway.
  • Install OpenShell 0.0.106.
  • Recreate a sandbox.

NemoClaw does not automatically truncate or change the sandbox name. Use the installed NemoClaw and OpenShell versions to create a replacement sandbox with a compatible name. Then use Transfer State Manually to move the required state into the replacement sandbox. After you verify the transferred state, destroy the incompatible sandbox with the installed NemoClaw and OpenShell versions. Then rerun the installer. If you manually retired the gateway, restore the previously installed OpenShell version and its gateway before you transfer the sandbox state.

$curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash
$nemo-deepagents upgrade-sandboxes --check

The default installer preserves a newer installed NemoClaw release. When a NemoClaw CLI is installed, it also stops without changing the CLI when it cannot verify the maintained lkg release. To intentionally install an older release from a fixed source, select its version tag or full commit SHA with NEMOCLAW_INSTALL_TAG.

$curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=vX.Y.Z bash

If a support workflow asks you to pass the maintained tag explicitly, clear any inherited NEMOCLAW_INSTALL_REF and set NEMOCLAW_INSTALL_TAG on the bash side of the install pipeline. The same downgrade protection applies to an explicit lkg selection.

$curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_INSTALL_REF= NEMOCLAW_INSTALL_TAG=lkg bash

During the automatic upgrade path, the installer prepares the current NemoClaw CLI without replacing OpenShell and requires a fresh backup of every registered sandbox. If any sandbox is skipped or fails, the installer exits before it changes the gateway. After backup, the installer requires any existing OpenShell executable it will use to report a version, then compares that version with the supported range declared by the prepared current source. It retires the running gateway before replacing an out-of-range OpenShell installation, keeps the gateway when the installed version is supported, and stops without retiring it when the installed version or supported range cannot be validated. If OpenShell lifecycle commands cannot retire the gateway, the installer can use verified service or PID-file evidence on Linux and macOS. For the default gateway on port 8080, Linux first checks verified systemd service evidence for nemoclaw-openshell-gateway.service, while macOS first checks verified Homebrew gateway service evidence. On either platform, the installer can then use a verified NemoClaw-managed gateway PID file for the configured gateway port. If macOS cannot obtain a conclusive lsof listener observation for a stale PID, the installer stops retirement and preserves the PID file, OpenShell registration, and every sandbox backup. After the service or PID-file check confirms the gateway process is stopped, the installer tries to remove the selected OpenShell registration. If registration removal fails, the installer warns that onboarding will replace the stale registration. If neither fallback can verify and stop the process, the installer stops after backup, preserves every sandbox backup, and reports the failed operation. After the automatic path retires an out-of-range gateway, it forces installation of the OpenShell version pinned by the prepared source before sandbox recovery. This mandatory installation applies to source and managed install modes and cannot remain deferred after gateway retirement. If the forced installation fails, the installer does not stage a gateway service or start sandbox recovery, preserves the backups, and reports how to retry.

Resume a Manually Prepared Upgrade

Use this path only after you back up every registered sandbox and retire the gateway. Set NEMOCLAW_GATEWAY_PORT to the port for the gateway that you retired. Set NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 on the bash side of the install pipeline when you rerun the installer.

$curl -fsSL https://www.nvidia.com/nemoclaw.sh | NEMOCLAW_GATEWAY_PORT=<selected-port> NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1 bash

Set NEMOCLAW_GATEWAY_PORT=8080 when you prepared the default gateway. Omitting it selects port 8080 only when no automatic alternate-port marker exists, so an explicit value prevents a previously recorded alternate environment from receiving the prepared upgrade. NEMOCLAW_OPENSHELL_UPGRADE_PREPARED asserts that you completed those steps, so the installer skips the repeated backup and gateway-retirement phase before it checks whether OpenShell is installed or whether its version is in range. It reuses the latest backups, forces installation of the OpenShell version pinned by the prepared source, and starts recovery only after that installation succeeds. If the installation fails, rerun the same install-pipeline command to preserve NEMOCLAW_GATEWAY_PORT and NEMOCLAW_OPENSHELL_UPGRADE_PREPARED. Outside these forced paths, an in-range source checkout preserves an existing OpenShell installation and installs OpenShell only when it is missing.

Reconcile Registered Sandboxes

After the host CLI and OpenShell update, the installer runs nemo-deepagents upgrade-sandboxes --auto to reconcile the existing sandboxes.

During installer-driven recovery, each stale or non-Ready sandbox requires a validated latest backup. The backup’s sandbox and agent identities must match the registry. The registry must also contain positive evidence that NemoClaw managed the sandbox image. If the replacement gateway reports a stale sandbox as Ready or Running, the installer reuses the validated pre-upgrade backup. It does not attempt another backup from the replaced legacy runtime.

The installer attempts every eligible recovery and exits with a nonzero status if any recovery fails. Recovery preserves each sandbox’s recorded GPU selection and runs before generic host admission. Fresh onboarding and required DGX Station reconciliation still require host admission.

Interpret Automatic Recovery Results

Use the installer’s completion message to distinguish these outcomes:

  • Confirmed recovery: The installer reports that existing sandboxes were recovered and upgraded. It skips generic onboarding when no Station reconciliation remains necessary.
  • Unconfirmed recovery output: The recovery command succeeded, but the installer could not inspect its output. The installer reports Installation completed with warnings and skips generic onboarding. Inspect the output of the read-only nemo-deepagents upgrade-sandboxes --check command before further action. A successful recovery command alone does not confirm that every sandbox recovered.
  • Sandbox not found on its recorded gateway: The recovery pass exits 0 but reports the sandbox as not found rather than recovered. The installer reports Installation completed with warnings and skips generic onboarding. A prior uninstall could have removed the gateway and Docker image while preserving sandboxes.json.

If you choose to remove a stranded sandbox, follow the cleanup sequence below.

Destroying a sandbox can remove its persistent state. Preserve any required backups before cleanup. A local-only forced cleanup does not confirm that the OpenShell sandbox was deleted.

  1. Check the recorded gateway with nemo-deepagents <name> status. This command can attempt recovery of the recorded gateway.
  2. Retry ordinary deletion with nemo-deepagents <name> destroy.
  3. If the gateway remains unavailable, review the destroy --force fallback limits before using nemo-deepagents <name> destroy --force. When gateway deletion is unconfirmed, this fallback removes only local state; the OpenShell sandbox can still exist. Managed MCP ownership or a sandbox-deletion timeout can prevent this fallback.
  4. If the gateway returns, verify or remove any remaining OpenShell sandbox before running nemo-deepagents onboard. Do not infer gateway-side deletion from removal of the local record.

Use the Manual Update Flow

Create a snapshot before a manual update.

$nemo-deepagents <sandbox-name> snapshot create --name pre-upgrade
$nemo-deepagents update --yes
$nemo-deepagents upgrade-sandboxes --check

Use nemo-deepagents upgrade-sandboxes --check to read sandbox state without starting, recovering, or selecting a gateway. The command exits nonzero when it reports stale state, an unknown version, a backup recovery condition, or a sandbox missing from its recorded gateway. Inspect each diagnostic before you select a remediation or automate the next action. When all your registered sandboxes resolve to one recorded gateway, the command queries that gateway instead of the gateway for your current NEMOCLAW_GATEWAY_PORT.

Each rebuild destroys the old container and creates a new one while preserving the manifest-defined workspace or agent state that NemoClaw knows how to snapshot. upgrade-sandboxes --check can report a sandbox as stale because the running agent version is behind, because the managed NemoClaw image fingerprint differs from the current CLI, or both.

Custom-image sandboxes created with --from <Dockerfile> are not marked stale solely by image fingerprint, so an upgrade check does not accidentally replace them with the default image. Runtime changes outside the manifest-defined state paths, such as packages installed manually in the running container, are not preserved.