Create and Restore Snapshots

View as Markdown

NemoClaw snapshots preserve manifest-defined sandbox state before destructive or state-changing operations. They are the preferred backup and restore path.

When to Create a Snapshot

  • Before running nemohermes <name> destroy.
  • Before major NemoClaw version upgrades.
  • Periodically, if you have invested time customizing your agent or paired messaging channels.

Understand Snapshot Contents

Snapshots capture the manifest-declared snapshot state directories and store them in ~/.nemoclaw/rebuild-backups/<name>/. Agent manifests can also declare durable top-level state files. Treat snapshot directories as private local data.

Before NemoClaw marks a snapshot complete, it strips recognized credential values from copied JSON, YAML, and .env files. It preserves OpenShell credential placeholders so rebuild can reattach the host-side provider. If NemoClaw cannot sanitize a copied configuration or environment file, it omits that file from the snapshot. If it cannot remove the unsafe file, snapshot creation returns an error. It deletes the incomplete backup when cleanup succeeds and reports when the backup remains. This sanitization uses an isolated python3 helper on POSIX hosts to keep reads, replacements, and removals anchored to opened directory descriptors. If a copied file or parent directory changes identity during the operation, snapshot creation fails closed instead of following the changed path.

Hermes snapshots include SOUL.md, the Web Dashboard profile under .hermes/profiles/dashboard-home/, the SQLite database behind .hermes/state.db, and the default kanban board in .hermes/kanban.db. On the first startup after this layout change, NemoClaw moves an existing .hermes/dashboard-home/ directory when the canonical profile is absent or empty and the legacy path is not a symlink. Startup sets the canonical dashboard profile directory to mode 0700, whether it migrates legacy state or reuses an existing destination. During normal startup, NemoClaw refuses unsafe paths and does not merge two populated profile directories. The default-profile snapshot also includes cron execution history in .hermes/runtime/cron-executions.db and Discord replay state in .hermes/gateway/discord_message_recovery.db. NemoClaw captures cron job definitions from .hermes/cron and user-authored cron scripts from .hermes/scripts as directory state. NemoClaw uses SQLite’s online backup API and restores these databases through SQLite instead of copying live raw database files. After it replaces a database, NemoClaw opens a write transaction against the result and fails the restore when the database cannot be written. Named-profile cron and Discord databases under .hermes/profiles/<name>/ use raw directory capture and can be inconsistent if a write overlaps the snapshot.

Kanban backup is limited to the backward-compatible default board in kanban.db. Named boards, attachments, worker logs, scratch workspaces under .hermes/kanban/, and external directory or worktree targets are not included; back up that state separately.

The dashboard profile includes MEMORY.md and USER.md. The Hermes state database can contain session metadata and message history needed for a faithful restore.

Snapshots preserve sandbox registry metadata that affects rebuild behavior, including custom policy presets applied with policy add --from-file or policy add --from-dir and baseline network policy entries excluded with policy exclude. When you restore a snapshot, NemoClaw replays those recorded custom presets with their stored YAML content, so you do not need the original preset files on disk, and rebuild continues to apply the recorded baseline exclusions.

The target sandbox’s current agent manifest remains authoritative for directory and state-file restore behavior. NemoClaw rejects the restore when the snapshot’s agent, config directory, any snapshot directory, state-file path, or state-file strategy conflicts with that manifest. Restore limits directory cleanup to state directories authorized by both the snapshot and the current manifest. It preserves target-only directories and directories whose backup failed.

For managed images, NemoClaw applies the current manifest’s managed config merge rules by default and does not fall back to whole-file replacement. For Deep Agents targets, whole-file config replacement is limited to sandboxes created from a custom Dockerfile.

Create and List Snapshots

$nemohermes my-assistant snapshot create
$nemohermes my-assistant snapshot list

snapshot list prints a table of version, name, timestamp, and path. NemoClaw computes versions (v1, v2, through vN) from timestamp order, so vN is always the newest snapshot.

snapshot create requires shields to be down. Snapshot creation and restore share the per-sandbox transition lock with the shields auto-restore timer.

If a timed shields-down window expires during snapshot work, the deadline gate blocks new mutations and waits for the exact snapshot owner to finish without signaling it. Snapshot work does not bypass recovery for an expired shields-down window. Follow Timed Shields Windows to correct state-directory failures or complete exact-generation recovery before you rerun nemohermes <name> snapshot create.

Tag a snapshot with a human-readable label:

$nemohermes my-assistant snapshot create --name before-upgrade

When a directory or state file cannot be captured, snapshot create reports the failed items, removes the incomplete snapshot, and exits nonzero. snapshot list shows no new entry, so a later restore cannot select a capture that never completed. If removal fails, the command reports the listed snapshot path. Remove that directory manually before you run snapshot restore because the incomplete capture remains selectable.

Restore a Snapshot

Restore the latest snapshot:

$nemohermes my-assistant snapshot restore

Pass an exact version, name, or timestamp to select a specific snapshot. Use the exact timestamp from snapshot list; a timestamp prefix does not select a snapshot.

$nemohermes my-assistant snapshot restore v3
$nemohermes my-assistant snapshot restore before-upgrade
$nemohermes my-assistant snapshot restore 2026-04-14T09-40-09-760Z

Post-restore policy reconciliation is best-effort. NemoClaw warns and continues the remaining restore steps in these cases:

  • NemoClaw cannot verify whether a custom policy owns the live observability-otlp-local policy entry.
  • The built-in observability-otlp-local policy preset has drifted or cannot be inspected.
  • NemoClaw cannot add or remove a recorded policy preset.

The live network policy can then retain unwanted egress or omit expected egress until you repair the named preset. After a warning, run nemohermes <name> policy list. Confirm that the named preset is recorded in the sandbox registry and active on the gateway, or absent from both.

A running Hermes gateway keeps serving its pre-restore state databases until it reopens them. After a restore that includes Hermes state databases, the CLI prints a reminder to restart the gateway. Run nemohermes <name> gateway restart to make the gateway open the restored databases.

To clone a snapshot into a different sandbox name, pass --to <name>. If the destination sandbox already exists, NemoClaw refuses to overwrite it unless you pass --force:

$nemohermes my-assistant snapshot restore before-upgrade --to my-assistant-clone
$nemohermes my-assistant snapshot restore before-upgrade --to my-assistant-clone --force --yes

Cross-sandbox restore from a stopped source is available for Docker- and VM-driver sandboxes. For a stopped source, its registry entry must record both the sandbox image and a complete inference route; NemoClaw creates the destination from the recorded image. NemoClaw stops before creating or replacing the destination when either record is missing, and directs you to run nemohermes onboard when no image is recorded. For a Kubernetes-driver source, the pod image must remain resolvable through its gateway.

For dashboard-enabled agents, NemoClaw allocates the destination sandbox its own dashboard port instead of reusing the source port. If no port is available, restore stops before deleting an existing --force destination.

The force-overwrite path restores and verifies lockdown on a destination with an active shields timer, then revokes that timer before it deletes the destination. It clears the remaining local shields state only after deletion succeeds, before a same-name replacement is created.

Restore Agent Configuration Safely

The nemohermes <name> rebuild command uses the same snapshot mechanism automatically. NemoClaw rejects unsafe symlinks and hard links inside sandbox state during backup creation before they can enter a snapshot.

Credential-bearing Hermes files such as auth.json are intentionally excluded from snapshots. NemoClaw-regenerated Hermes config files, including config.yaml and .env, are also excluded. NemoClaw recreates model, provider, and messaging credentials from host-side onboarding and OpenShell provider state during rebuild.

If a Hermes rebuild cannot validate or release its NemoClaw cron restore gate, NemoClaw preserves the state backup. If the rebuild already accepted the replacement sandbox, it also preserves the replacement journal. New Hermes turns and cron dispatch remain blocked while the gate exists.

Do not manually remove the root-owned cron restore marker. Removing it bypasses validation of the restored cron jobs and scripts. Correct the reported restore problem, then run nemohermes <sandbox-name> recover. Recovery validates the restored cron tree before it clears the NemoClaw gate. If an independent Hermes operator drain exists, recovery leaves it active. After recovery succeeds, rerun rebuild with the same replacement settings so NemoClaw can retire the replacement journal.

After a rebuild restores dashboard-home or profiles, NemoClaw reruns the dashboard state migration before it reports the restore as complete. During rebuild restore, NemoClaw moves disjoint top-level entries from the legacy dashboard directory without replacing entries in the canonical profile. If an entry collides or migration otherwise fails, NemoClaw marks the restore incomplete instead of reporting success.

Back Up Every Registered Sandbox

Run nemohermes backup-all before broad maintenance such as nemohermes update, nemohermes upgrade-sandboxes, or an OpenShell gateway migration.

$nemohermes backup-all

backup-all walks the sandboxes registered on the host, creates a snapshot for each eligible running or temporarily started sandbox, and stores the snapshot bundles under ~/.nemoclaw/rebuild-backups/<name>/. If a registered docker-driver sandbox’s container is stopped, backup-all starts the container for the duration of the backup and returns it to its stopped state afterward. If the container cannot be returned to the stopped state, the backup run fails and reports that the container was left running. If a sandbox is not running and its container cannot be started this way, start the sandbox or its container and rerun nemohermes backup-all.

For each eligible sandbox, backup-all holds one lifecycle transaction through the complete backup. Within that transaction, it starts a stopped container when required, opens a 30-minute shields-down window when the sandbox starts with Shields up, copies sandbox state, restores the previous Shields state, and returns any container it started to the stopped state. A sandbox that starts with Shields down remains down. If the timer expires during the transaction, the deadline gate blocks new mutations and waits for the exact backup owner to finish without signaling it. An initial lock or unlock failure marks that sandbox as failed, and backup-all continues with the next sandbox. NemoClaw attempts to restore the previous Shields state before it processes the next sandbox, including when the backup fails. If lockdown cannot be restored, backup-all stops and does not process the remaining sandboxes. Correct the reported issue, run the printed nemohermes <name> shields up command, and rerun nemohermes backup-all.

When a backup fails, NemoClaw identifies the affected state item and reports permission denied, tar read error, or absent after extraction when available. Use nemohermes <name> snapshot list and nemohermes <name> snapshot restore to inspect or restore one sandbox’s bundles later.