Troubleshooting
This page covers common installation, onboarding, and runtime issues, along with resolution steps.
The diagnostic commands on this page assume nemoclaw is on your PATH (re-source your shell profile after an nvm- or fnm-managed install) and that your user can reach the Docker socket — either as a member of the docker group or by running the Docker commands with sudo.
Get Help
If your issue is not listed here, join the NemoClaw Discord channel to ask questions and get help from the community. You can also file an issue on GitHub.
Installation
nemoclaw not found after install
If you use nvm or fnm to manage Node.js, the installer may not update your current shell’s PATH.
The nemoclaw binary is installed but the shell session does not know where to find it.
Run source ~/.bashrc (or source ~/.zshrc for zsh), or open a new terminal window.
When installing from a source checkout with npm install, NemoClaw first tries npm link.
If the global npm prefix is not writable, it writes a managed shim to ~/.local/bin/nemoclaw instead.
Add ~/.local/bin to your PATH if the command is still not found.
Source-checkout installs also bootstrap OpenShell when it is missing before running preflight.
If a source install still reports that openshell is not available, re-run the installer from the repository root and check that ~/.local/bin is on your PATH.
Installer fails on unsupported platform
The installer checks for a supported OS and architecture before proceeding. If you see an unsupported platform error, verify that you are running on a tested platform listed in the Container Runtimes table in the quickstart guide.
Node.js version is too old
NemoClaw requires Node.js 22.19 or later. If the installer exits with a Node.js version error, check your current version:
If the version is below 22.19, install a supported release. If you use nvm, run:
Then re-run the installer.
Image push fails with out-of-memory errors
The sandbox image is approximately 2.4 GB compressed. During image push, the Docker daemon, k3s, and the OpenShell gateway run alongside the export pipeline, which buffers decompressed layers in memory. On machines with less than 8 GB of RAM, this combined usage can trigger the OOM killer.
If you cannot add memory, configure at least 8 GB of swap to work around the issue at the cost of slower performance.
Docker is not running
The installer and onboard wizard require Docker to be running. If you see a Docker connection error, start the Docker daemon:
On macOS with Docker Desktop, open the Docker Desktop application and wait for it to finish starting before retrying.
Docker permission denied on Linux
On Linux, if the Docker daemon is running but you see “permission denied” errors, your user may not be in the docker group.
The installer can add your user to the group, but Linux does not activate that membership in the current shell automatically.
Add your user and activate the group in the current shell:
Docker group access
NemoClaw needs Docker access.
On personal Linux development machines, adding your user to the docker group is the standard way to run Docker without sudo.
Members of the docker group can control the daemon with root-level impact, so grant this access only to trusted local accounts; on shared or managed systems, use your organization’s approved Docker access path.
For background, review Docker’s daemon attack surface guidance.
Then retry nemoclaw onboard.
If the installer stopped after printing newgrp docker, run that command and then re-run the installer:
Installer reports Docker access outside the docker group
On Linux, the installer may report that Docker is reachable even though your user is not in the docker group.
This means the host grants Docker daemon access through another path, such as a custom DOCKER_HOST, socket ACL, or managed runtime policy.
NemoClaw can continue when docker info works, but the diagnostic explains why a negative Docker-permission test will not reproduce on that host.
Check the Docker access path before relying on the host as a clean permission baseline:
macOS first-run failures
The two most common first-run failures on macOS are missing developer tools and Docker connection errors.
To avoid these issues, install the prerequisites in the following order before running the NemoClaw installer:
- Install Xcode Command Line Tools (
xcode-select --install). These are needed by the installer and Node.js toolchain. - Install and start a supported container runtime (Docker Desktop or Colima). Without a running runtime, the installer cannot connect to Docker.
docker is missing after installing Colima
Homebrew Colima does not install the Docker CLI binary.
If you install only Colima, colima start can succeed while later docker commands fail with command not found.
Install both packages, start Colima with enough resources for the sandbox image build, and verify Docker before onboarding:
Permission errors during installation
The NemoClaw installer does not require sudo or root.
It installs Node.js via nvm and NemoClaw via npm, both into user-local directories.
The installer also handles OpenShell installation automatically using a pinned release.
If you see permission errors during installation, they typically come from Docker, not the NemoClaw installer itself. Docker must be installed and running before you run the installer, and installing Docker may require elevated privileges on Linux.
npm install fails with permission errors
If npm install fails with an EACCES permission error, do not run npm with sudo.
Instead, configure npm to use a directory you own:
Add the export line to your ~/.bashrc or ~/.zshrc to make it permanent, then re-run the installer.
Installer fails on NVIDIA Jetson
The installer auto-detects NVIDIA Jetson devices (Orin and Thor) and applies required host configuration before the normal install flow.
If the Jetson setup step fails, verify that you have sudo access and that Docker is installed and running.
For JetPack 6 (L4T 36.x), the setup switches iptables to legacy mode and adjusts the Docker daemon configuration.
For JetPack 7 (L4T 38.x / Thor), only bridge netfilter and sysctl settings are applied.
For JetPack 7 (L4T 39.x), bridge netfilter is loaded only when the host is missing it.
Some R39 images already ship with br_netfilter configured and are left untouched.
On affected R39 hosts, the installer prints loading br_netfilter (required by k3s inside the OpenShell gateway).
Without this fix, sandbox pods fail DNS resolution against the in-cluster service and the onboard Setting up OpenClaw inside sandbox step times out.
If the L4T version is not recognized, the setup step is skipped and the installer continues normally.
DNS resolution from inside docker fails (corporate firewall)
Some corporate networks block outbound UDP port 53 to public DNS servers and force all host name resolution through DNS over TLS on TCP port 853. Containers do not inherit the host’s DNS-over-TLS configuration, so the sandbox build’s npm ci step times out trying to resolve registry.npmjs.org against 1.1.1.1 or 8.8.8.8.
NemoClaw’s preflight runs a short docker run --rm busybox nslookup nemoclaw-dns-probe-<random>.invalid probe before starting the sandbox build. The fresh .invalid name should return NXDOMAIN through a working resolver, so cached answers cannot hide blocked DNS egress. When the probe confirms a DNS failure, onboarding stops with platform-specific remediation instead of hanging for ~15 minutes and printing a cryptic Exit handler never called.
Use the preflight headline to choose the recovery path:
- If no DNS servers could be reached, Docker could not reach its configured resolver. Follow the platform-specific UDP port 53 and Docker DNS steps below.
- If the DNS server was reachable but rejected the query with
NXDOMAINorREFUSED, the resolver answered, so the UDP port 53 fix is not relevant. Check the resolver used by Docker, such as dnsmasq, Pi-hole, unbound, or systemd-resolved, and remove any forwarding rule, blocklist entry, or ACL that rejectsregistry.npmjs.org. If needed, configure Docker to use an organization-approved resolver that can resolve public names, restart Docker, and retry onboarding.
For an unreachable resolver, pick the matching platform path below, apply it, then re-run nemoclaw onboard.
- Linux with systemd-resolved. Add a
DNSStubListenerExtradrop-in pointing at the docker bridge gateway IP (the preflight prints the detected IP), then add the same IP to/etc/docker/daemon.jsonunderdns. Restartsystemd-resolvedanddocker. - macOS with Colima. Restart Colima with the corporate DNS address, for example
colima stop && colima start --dns <corp-dns-ip>. - macOS with Docker Desktop. Add the corporate DNS address to
~/.docker/daemon.jsonunderdns, then restart Docker Desktop. - Windows or WSL. Configure DNS in the Docker Desktop settings GUI, or apply the Linux fix above when running native docker inside WSL.
Verify the fix worked:
When the lookup returns an answer, retry onboarding.
Direct DNS lookups fail in a Docker-driver GPU sandbox
Egress covered by OpenShell network policies resolves destinations through the gateway. A direct DNS lookup inside the agent network namespace can fail on Docker-driver GPU hosts such as DGX Spark even when policy-covered inference, messaging, and search work normally. Direct in-sandbox DNS depends on Docker and the host resolver and is not a supported NemoClaw network-policy path, so NemoClaw does not use it as a sandbox health check.
Run a manual lookup only when you are diagnosing a custom tool that performs its own DNS resolution:
If the host alias resolves but the external name does not, Docker’s embedded resolver may be forwarding to an upstream DNS server that the sandbox bridge cannot use.
Do not treat this result as evidence that a policy-covered feature is unhealthy.
Test the affected inference, messaging, or search request through its normal policy path and inspect denied requests with openshell term.
If a custom tool requires direct DNS, configure the Docker daemon to use a resolver that containers can reach. On hosts with VPN or split-DNS software, use an upstream resolver that remains reachable from the Docker bridge, then recreate or rebuild the sandbox. Keep bridge networking enabled so the sandbox retains its normal Docker network isolation.
Host DNS resolution is blocked before provider validation
NemoClaw also checks that the host process can resolve the provider host before it starts NVIDIA provider validation.
A firewall rule that blocks host DNS traffic on port 53 can make later validation fail with curl: (6) Could not resolve host: integrate.api.nvidia.com even when container DNS probes look healthy.
Current onboarding stops earlier with a host DNS diagnostic and remediation hints.
Verify host DNS outside NemoClaw:
Fix the host firewall, VPN, or DNS policy so the host can resolve the provider endpoint, then rerun onboarding.
If you intentionally use a non-NVIDIA provider and need to bypass only this preflight, set NEMOCLAW_SKIP_HOST_DNS_PREFLIGHT=1.
Port already in use
The NemoClaw dashboard uses port 18789 by default and the gateway uses port 8080.
If another sandbox already owns the dashboard port, onboarding scans ports 18789 through 18799 and uses the next free port.
If all ports in that range are occupied, the error lists the owner for each port and suggests using --control-ui-port with a port outside the range.
On macOS, the port check also tries a privileged lsof probe without prompting for a password so root-owned listeners are detected before the sandbox build starts.
If a port becomes occupied after preflight but before openshell forward start runs, onboarding deletes the just-created sandbox and exits with a retry message instead of leaving a sandbox with an unreachable dashboard URL.
When a previous onboard, upgrade, or sandbox crash leaves a stale openclaw-gateway host process holding the dashboard port, nemoclaw onboard --fresh, nemoclaw <name> destroy (when destroying the last sandbox), and nemoclaw uninstall automatically sweep the dashboard port range and signal SIGTERM then SIGKILL to recover.
The sweep only targets processes owned by the current user whose command line matches openclaw-gateway or openshell forward markers, and skips dashboard ports owned by other live sandboxes.
If a non-NemoClaw process is already bound to the dashboard port or the gateway port, identify the conflicting process, verify it is safe to stop, and terminate it:
If the process does not exit, use kill -9 <PID> to force-terminate it.
Then retry onboarding.
Alternatively, override the conflicting port instead of stopping the other process.
Pass --control-ui-port with the desired dashboard port:
You can also set CHAT_UI_URL with the desired port:
Or set the port directly:
For an OpenShell gateway port conflict, set NEMOCLAW_GATEWAY_PORT to a free non-privileged port that does not overlap NemoClaw’s dashboard, vLLM, Ollama, or Ollama proxy ports:
Remote/headless hosts should keep the OpenShell gateway on loopback and bind the dashboard forward instead:
Use NEMOCLAW_DASHBOARD_BIND=0.0.0.0 again on later nemoclaw <sandbox> connect calls.
If the sandbox was originally created without remote bind, recreate it with the same onboard command plus --recreate-sandbox before connecting remotely.
Docker-driver gateways on OpenShell 0.0.85 reject NEMOCLAW_GATEWAY_BIND_ADDRESS=0.0.0.0 while gateway JWT auth is active.
Use NEMOCLAW_GATEWAY_BIND_ADDRESS=0.0.0.0 only on supported gateway modes and only when other hosts on the network should be able to reach the gateway.
Older-glibc gateway compatibility container
OpenShell 0.0.85 directly supports Linux hosts with glibc 2.28 or newer.
On an older trusted host, NEMOCLAW_OPENSHELL_GATEWAY_CONTAINER_PATCH=1 explicitly opts into NemoClaw’s compatibility container.
Leave it unset on supported hosts.
The compatibility container uses host networking and mounts the host Docker socket read-only. A read-only socket mount still permits privileged Docker API operations and can control the host, so do not enable this mode on an untrusted or shared host. The gateway remains loopback-bound, and startup fails closed unless the configured Unix socket answers as a Docker daemon. See the OpenShell gateway compatibility review for the unchanged container boundary and removal conditions.
Refer to Environment Variables for the full list of port overrides.
Running multiple sandboxes simultaneously
Each sandbox requires its own dashboard port.
If you onboard a second sandbox without overriding the port, onboarding uses the next free port in the 18789 to 18799 range.
onboard checks openshell forward list before starting a new forward, so a second onboard cannot silently take over the first sandbox’s port.
Assign a distinct port only when you want a specific value:
Each sandbox then has its own SSH tunnel and its own dashboard URL:
You can verify which tunnel belongs to which sandbox with:
nemoclaw list prints the recorded dashboard URL for each sandbox.
These dashboard ports are separate from the gateway-wide inference route.
A shared inference route conflicts with another sandbox
Refer to Use Shared Gateway Routes for route time-sharing, provider-global compatibility, and status drift fields.
If inference set reports a valid shared-route conflict, align the named sandbox records or remove a sandbox you no longer need.
If onboarding or connect reports a provider-global identity conflict, align the same-name provider’s custom endpoint, API family, and credential environment-variable name across the named sandboxes, or remove a conflicting sandbox you no longer need.
If the error names incomplete legacy custom-route metadata, back up and remove the affected sandbox, then re-onboard it with an explicit custom endpoint and API family. For an OpenAI-compatible route, replace the example endpoint, model, and sandbox name in this recovery sequence:
If the error names an invalid gateway binding, restore the affected row’s known-good gatewayName and gatewayPort metadata from a trusted backup; otherwise back up and remove the sandbox, then re-onboard it.
Do not guess or copy a binding from another sandbox because lifecycle commands use it to select the gateway.
Onboarding
Cgroup v2 errors during onboard
Older NemoClaw releases relied on a Docker cgroup workaround on Ubuntu 24.04, DGX Spark, and WSL2. Current OpenShell releases handle that behavior themselves, so NemoClaw no longer requires a Spark-specific setup step.
If onboarding reports that Docker is missing or unreachable, fix Docker first and retry onboarding:
Podman is not a tested runtime. If onboarding or sandbox lifecycle fails, switch to a tested runtime (Docker Desktop, Colima, or Docker Engine) and rerun onboarding.
Cluster fails with overlayfs snapshotter cannot be enabled on Docker 26+
Docker Engine 26 and later default fresh installations to the containerd image store, which exposes its layers via the overlayfs snapshotter rather than the legacy overlay2 graph driver.
The k3s server inside the OpenShell cluster image needs to mount its own overlay filesystem on top, and the kernel rejects nesting two non-trivial overlay mounts.
The cluster container then loops with:
This is a Docker default-driver change, not a NemoClaw or OpenShell regression.
The same hardware uses the legacy overlay2 driver and is unaffected when it runs Docker 25 or earlier, or any Docker version with the containerd image store disabled.
NemoClaw detects the Docker 26+ containerd-snapshotter overlayfs configuration during onboarding and transparently builds a small drop-in replacement for the cluster image on the local Docker engine.
The patched image installs fuse-overlayfs and selects it as the k3s snapshotter, bypassing the kernel-level nested-overlay limitation.
No host configuration changes, sudo, or Docker restart required.
The auto-fix runs once per OpenShell version on the affected host.
Subsequent onboarding runs reuse the cached patched image.
Hosts without the conflict (Driver: overlay2 in docker info, macOS Docker Desktop, or Linux installations that disable the containerd image store) see no change in behavior.
Override knobs:
NEMOCLAW_DISABLE_OVERLAY_FIX=1: skip the auto-fix and run against the unmodified upstream cluster image. Useful for diagnosis or when you have already applied the manual workaround below.NEMOCLAW_OVERLAY_SNAPSHOTTER=native: build the patched image with k3s’snativesnapshotter instead offuse-overlayfs. Thenativesnapshotter copies image layers instead of overlaying them, so it uses more disk but does not depend on FUSE. Default isfuse-overlayfs.
If you prefer to disable the new Docker storage driver instead of running the patched image, edit /etc/docker/daemon.json:
Then restart Docker (sudo systemctl restart docker) and re-run nemoclaw onboard.
This restores the legacy overlay2 driver host-wide, which kills any other running containers.
Prefer the auto-fix unless you need the change for unrelated reasons.
Switching storage drivers also rebuilds the entire local image graph: previously-pulled images become unusable and Docker re-pulls them on first reference, so expect a cold cache and additional disk usage right after the restart.
OpenShell version above maximum
Each NemoClaw release validates against a range of tested OpenShell versions.
If the installed OpenShell version exceeds the configured maximum, nemoclaw onboard exits with an error:
Upgrade NemoClaw to a version that supports your OpenShell release, or install a supported OpenShell version from the OpenShell releases page.
For fresh installs, NemoClaw passes the blueprint range to install-openshell.sh and resolves a compatible published OpenShell release before downloading.
If GitHub release metadata is unavailable, the script uses its bundled fallback pin and the post-install gate still enforces the configured range.
Sandbox build fails during OpenClaw plugin install
During sandbox creation, the OpenClaw image setup can install managed plugins for selected features such as web search or diagnostics.
If the build reaches openclaw plugins install and the npm registry or ClawHub is blocked, NemoClaw classifies that narrow failure and prints a policy hint instead of only generic resume guidance.
Brave Search uses an external OpenClaw plugin and can reach this install path.
Tavily ships with the pinned OpenClaw runtime, so NemoClaw verifies the bundled extension instead of installing a separate Tavily package.
Check that the active policy and host network allow the npm registry and ClawHub endpoints needed by the plugin, or disable the feature that requested the plugin. For example, if the plugin is for web search, disable that feature and resume onboarding:
If you want the feature, fix the network or policy path first, then resume onboarding:
Web search verification reports a warning
When web search is enabled, onboarding checks the selected agent configuration and sends a real search request through the sandbox egress path. The verification is best effort, so a failed check prints a warning and lets onboarding finish.
First confirm that the provider credential and matching policy preset exist.
Look for <name>-brave-search with the brave preset or <name>-tavily-search with the tavily preset.
Do not replace an openshell:resolve:env:<KEY> value in the sandbox configuration with a raw API key.
Confirm that OpenClaw reports the provider selected during onboarding.
The provider should be brave or tavily and enabled should be true.
If the provider is wrong, rerun onboarding with NEMOCLAW_WEB_SEARCH_PROVIDER=brave or tavily and the matching BRAVE_API_KEY or TAVILY_API_KEY.
Rerunning onboarding with a different provider recreates the sandbox because the provider configuration and credential attachment are build-time inputs.
NemoClaw validates the replacement key before it removes the existing sandbox, then backs up and restores the supported workspace state during recreation.
If the configuration is correct but the egress probe fails, keep the matching preset applied and inspect the blocked request with openshell term before widening any policy rule.
Sandbox containers cannot reach the gateway
On native Linux Docker-driver hosts, nemoclaw onboard verifies the route that sandbox containers use to reach the OpenShell gateway.
If a host firewall blocks that path, onboarding exits with output like:
Apply the ufw command printed by onboarding, then rerun onboarding.
If the message does not include a subnet, derive it from the OpenShell Docker network:
Custom OpenClaw image creates without a gateway or dashboard
nemoclaw onboard --from <Dockerfile> treats the supplied Dockerfile as the complete sandbox image rather than adding it on top of the stock managed runtime.
If deployment verification cannot reach the gateway, NemoClaw checks for /tmp/gateway.log, /usr/local/bin/nemoclaw-start, and /sandbox/.openclaw/openclaw.json in the custom sandbox.
When all three paths are absent, the CLI reports that the image lacks the NemoClaw-managed OpenClaw runtime and does not suggest repeated dashboard port-forward retries.
This failure commonly occurs when the custom Dockerfile starts from ghcr.io/nvidia/nemoclaw/sandbox-base alone because that image is an intermediate dependency image.
Rebuild the custom image from the full stock Dockerfile and source context for the same NemoClaw release. For the version-pinned plugin workflow, refer to Install OpenClaw Plugins.
If the sandbox is unreachable or the managed runtime paths are present, NemoClaw retains the existing generic gateway-log and host OpenShell-log guidance because the base-only failure is not proven.
connect exits because the gateway is down
nemoclaw <name> connect checks the OpenShell gateway before it tries dashboard forwarding, SSH, or inference repair.
If the gateway is not reachable, the command exits early and prints recovery guidance.
Resume onboarding so NemoClaw recreates or reconnects the managed gateway, then retry:
Run nemoclaw status for a broader gateway health report.
Sandbox container reports (unhealthy) while the agent gateway process is still alive
The in-sandbox OpenClaw gateway can drop its HTTP listener while its process stays alive.
A restart-class configuration change makes the gateway restart itself in place, and if that restart fails the process parks with no listener (/tmp/gateway.log shows gateway startup failed: ... Process will stay alive).
Docker then marks the container (unhealthy) even though pgrep still finds the gateway.
NemoClaw prevents and self-heals this:
- The generated sandbox config pins
gateway.reload.modetohot, so configuration changes never make the gateway restart itself out from under the sandbox supervisor. - A serving watchdog inside the sandbox kills a gateway that stops listening after it has served, and the supervisor relaunches it (look for
[gateway-watchdog]lines innemoclaw <name> logs).
Because of the hot pin, restart-class configuration changes made inside the sandbox log config reload requires gateway restart; hot mode ignoring and do not take effect until the gateway restarts.
For example, openclaw plugins install logs that message until the gateway restarts.
Apply them with a supervised restart:
or rebuild the sandbox for changes that affect provisioning:
Invalid sandbox name
Sandbox names must be lowercase, start with a letter, contain only letters, numbers, and internal hyphens, and end with a letter or number.
The CLI rejects names that do not match these rules.
It prints a Try: <suggested-slug> recovery line whenever it can derive a valid lowercase, hyphen-separated form from the input, so passing --name MyAssistant reports Try: myassistant and you can rerun with the suggested slug.
Names that collide with global CLI commands are also rejected.
Reserved names include onboard, list, deploy, setup, start, stop, status, debug, uninstall, credentials, and help.
Using a reserved name would cause the CLI to route to the global command instead of the sandbox.
If the name does not match these rules or is reserved, the wizard exits with an error.
Choose a name such as my-assistant or dev1.
Sandbox creation fails on DGX
On DGX machines, sandbox creation can fail if the gateway’s DNS has not finished propagating or if a stale port forward from a previous onboard run is still active.
Run nemoclaw onboard to retry.
The wizard cleans up stale port forwards and waits for gateway readiness automatically.
GPU setup fails with a placeholder GPU name
On Windows or WSL hosts, some systems report a placeholder display adapter name even when no NVIDIA GPU firmware is present. NVIDIA NIM and GPU-backed sandbox setup require a real NVIDIA GPU. If NemoClaw rejects the detected GPU name during preflight, select a CPU or remote inference provider, or move the setup to a host with a supported NVIDIA GPU and current drivers.
Jetson/Tegra hosts support sandbox GPU passthrough through the compatibility route.
Onboarding detects those hosts separately and propagates the supplementary groups required for /dev/nvmap and /dev/nvhost-*; if that path fails, follow the Jetson/Tegra compatibility guidance below instead of treating a missing nvidia-smi result as a placeholder adapter.
Colima socket not detected (macOS)
Newer Colima versions use the XDG base directory (~/.config/colima/default/docker.sock) instead of the legacy path (~/.colima/default/docker.sock).
Some installations expose a top-level Colima socket at ~/.colima/docker.sock.
NemoClaw checks all three paths.
If neither is found, verify that Colima is running:
Sandbox build is slow or hangs (under-provisioned container runtime)
Default Colima ships with 2 vCPU and 2 GiB of memory, which is not enough headroom for the BuildKit-driven sandbox image build. On macOS Apple Silicon, the build can stall part-way through with no progress and no error, leaving the wizard waiting indefinitely.
Preflight inspects docker info for NCPU and MemTotal and prints a warning when the runtime falls below 4 vCPU or 8 GiB.
In interactive onboarding, the warning prompt defaults to abort, so pressing Enter stops the run before the sandbox build reaches the likely stall point.
Type y only when you intentionally want to continue on the smaller runtime.
Non-interactive onboarding prints the warning and continues.
On Colima, raise the resources before re-running onboard:
On Docker Desktop, raise CPU and memory limits in Settings → Resources, then apply and restart.
To silence the warning when the host is intentionally small, set NEMOCLAW_IGNORE_RUNTIME_RESOURCES=1 before running nemoclaw onboard.
Re-onboard fails because port 18789 is held by SSH
After destroying a sandbox and gateway, the SSH port-forward process for the dashboard can be left running.
Re-running onboard then fails preflight with Port 18789 is not available. Blocked by: ssh.
Current NemoClaw detects this case and kills the orphaned SSH process automatically before retrying the port check. If you see the error on an older release, identify the SSH process and terminate it manually:
Then re-run nemoclaw onboard.
Updated messaging token is not picked up
Re-running nemoclaw onboard --non-interactive with a new TELEGRAM_BOT_TOKEN, DISCORD_BOT_TOKEN, or SLACK_BOT_TOKEN previously reported success while the sandbox kept polling with the old credential.
Current NemoClaw stores SHA-256 hashes of messaging credentials in the sandbox registry at creation time and detects when a token has changed.
When rotation is detected, NemoClaw automatically backs up workspace state, deletes the sandbox, recreates it with the new credential, and restores the backup.
If you suspect a sandbox is still using a stale token, re-run onboarding so the credential check runs:
Sandbox creation killed by OOM (exit 137)
On systems with 8 GB RAM or less and no swap configured, the sandbox image push can exhaust available memory and get killed by the Linux OOM killer (exit code 137).
NemoClaw automatically detects low memory during onboarding and prompts to create a 4 GB swap file.
If this automatic step fails or you are using a custom setup flow, create swap manually before running nemoclaw onboard:
Previous onboarding session failed
If a previous nemoclaw onboard attempt fails partway through (for example, a provider or inference-setup step reporting an error), NemoClaw records the failure in ~/.nemoclaw/onboard-session.json.
When you re-run the installer, it detects the failed session and does not silently retry it. Silent retry would loop on the same failure if your original choice, such as an unreachable provider, was the cause.
- In an interactive terminal, the installer prompts whether to resume the failed session or start fresh.
Press
R(or Enter) to retry the same session, orfto discard it and make fresh choices. - In non-interactive mode (piped
curl | bashwithNEMOCLAW_NON_INTERACTIVE=1, CI, scripts), the installer refuses and exits with a non-zero status so a scripted re-run cannot loop. You must opt in to one of two paths explicitly:
Start over with new choices to discard the recorded session and provider/model selection.
Or use environment variables instead.
Set them on the bash side of the pipe because only the right-hand process inherits them.
Retry the same session.
This is only useful if the original failure was transient, for example a network blip or a stopped Docker daemon, and not a wrong provider choice:
OpenClaw resume does not repeat completed non-secret sandbox, web search, messaging, or resource choices.
Resume also reuses registered web search and messaging credentials when the same onboarding session recorded their successful OpenShell registration and OpenShell still reports the exact expected name, type, and credential keys.
If the session lacks that registration receipt, the provider is missing, or its binding does not match, interactive resume requests the credential again; non-interactive resume preserves the completed choice, reports the required environment variable, and exits so you can export it before retrying nemoclaw onboard --resume.
As a last resort, you can also delete the session file directly and re-run the installer:
Kubernetes namespace not ready
If onboarding fails with Kubernetes namespace not ready, a previous failed or interrupted setup may have left stale OpenShell or NemoClaw state behind.
Clean up the failed installation before re-running the installer:
The normal uninstall path keeps user data under ~/.nemoclaw/, including sandbox registry metadata, backups, and saved credentials unless you explicitly remove them.
If nemoclaw uninstall reports that the local uninstall script is missing, follow the CLI’s security boundary: download the versioned NVIDIA/NemoClaw tag URL that it prints, inspect the script locally, run that local copy, and then retry the installer.
Runtime
OpenShell gateway and OpenClaw gateway startup order
NemoClaw uses two gateway layers for OpenClaw sandboxes:
- The OpenShell gateway runs on the host side and owns sandbox lifecycle, provider routes, port forwards, and
openshell sandbox list/statusqueries. - The OpenClaw gateway runs inside the sandbox container and serves the OpenClaw dashboard, agent API, and sub-agent WebSocket traffic.
Start and recover them in this order: container runtime, OpenShell gateway, sandbox container, then the in-sandbox OpenClaw gateway.
Do not start the OpenClaw gateway by hand before the OpenShell gateway is healthy. NemoClaw cannot select, inspect, or reconnect the sandbox until OpenShell can see the owning gateway.
If the host rebooted or the OpenShell gateway is down, first run:
The status command selects or starts the sandbox’s recorded OpenShell gateway when possible, then checks whether OpenShell can still see the sandbox.
If the sandbox container is present but stopped on a Docker-driver host, status can recover the labeled container and then re-query OpenShell.
After the sandbox is visible again, use nemoclaw <name> recover only for the in-sandbox OpenClaw gateway and host forwards.
Use nemoclaw <name> gateway restart when you intentionally need the in-sandbox gateway to reload supported runtime configuration.
Reconnect after a host reboot
After a host reboot, the container runtime, OpenShell gateway, and sandbox may not be running. Follow these steps to reconnect.
-
Start the container runtime.
- Linux: start Docker if it is not already running (
sudo systemctl start docker) - macOS: open Docker Desktop or start Colima (
colima start)
- Linux: start Docker if it is not already running (
-
Check sandbox state.
If the sandbox shows
Ready, skip to step 4. -
Recover the managed gateway (if needed).
If the sandbox is not listed or the command fails, let NemoClaw recover the managed gateway and sandbox registration:
Wait a few seconds, then re-check with
openshell sandbox list. On Docker-driver hosts, NemoClaw also looks for OpenShell-labeled sandbox containers when the gateway is healthy but reports the sandbox as missing. It can start a stopped labeled container, or restore the latest GPU-backup sibling container name and start it. -
Reconnect.
The gateway usually rotates its SSH host keys across a reboot.
connectdetects the resulting identity drift, prunes the staleopenshell-*entries from~/.ssh/known_hosts, and retries automatically. You do not need to editknown_hostsby hand or re-runnemoclaw onboardin this case. -
Start host auxiliary services (if needed).
If you use the cloudflared tunnel started by
nemoclaw tunnel start, start it again:OpenShell-managed channel messaging handles Telegram, Discord, Slack, WeChat, and WhatsApp at onboarding, not through a separate bridge process from
nemoclaw tunnel start. WeChat and WhatsApp are experimental. To pause a single bridge without destroying the sandbox, usenemoclaw <name> channels stop <channel>.
If the sandbox does not recover
If the sandbox remains missing after restarting the gateway, run nemoclaw <name> rebuild --yes while the local registry entry still exists.
The rebuild path uses the recorded sandbox metadata and the snapshot flow to preserve supported workspace and agent state.
If the sandbox was intentionally deleted and you want a clean setup instead, run nemoclaw <name> destroy to remove the stale local entry, then run nemoclaw onboard.
Create a snapshot first when the sandbox is reachable enough to back up state.
For details, refer to Create and Restore Snapshots.
gateway restart or recover reports privileged control unavailable
Built-in OpenClaw and Hermes lifecycle commands require a running direct sandbox container that belongs to the named NemoClaw registry entry.
The host uses registry-scoped privileged direct-container control to send an authenticated request to the controller for the live topology.
It does not fall back to ordinary openshell sandbox exec, SSH, or a manual in-sandbox relaunch.
Current built-in images support two direct-container shapes.
A direct root-entrypoint container uses the root PID 1 supervisor.
An OpenShell-managed container uses /opt/openshell/bin/openshell-sandbox as PID 1, exactly one nonroot nemoclaw-start supervisor, and the installed root-owned mode 0500 managed controller.
An arbitrary nonroot entrypoint that does not match that managed process shape fails with privileged control unavailable.
Kubernetes and other deployments without a matching direct container also fail closed with privileged control unavailable.
Run the lifecycle command from a supported direct-container deployment rather than trying to launch the gateway by hand.
On a direct-container deployment, first confirm that the sandbox is running:
When recover repairs a stopped built-in gateway, NemoClaw repeats the recovery action only for an exit status of 1 with blank stdout and a sole nonblank stderr line equal to SUPERVISOR_BUSY, with at most three controller attempts.
The same result is inconclusive during managed settle confirmation and can be probed again only within the configured settle window.
NemoClaw treats SUPERVISOR_UNAVAILABLE as terminal because it can report unreadable or untrusted supervisor state, ambiguous discovery, or a process-identity change.
It does not retry other status or output combinations.
SUPERVISOR_NOT_RUNNING is a separate result that requires two zero-supervisor scans with a stable PID 1 and does not enter that retry loop.
On a supported local Docker-driver sandbox with the legacy keepalive startup, it can authorize a container-identity-pinned recreation that commits only after managed health and settle checks pass.
To bypass that trusted recreation while troubleshooting, run NEMOCLAW_DISABLE_SUPERVISOR_RELAUNCH=1 nemoclaw <name> recover; NemoClaw leaves the container unchanged and returns rebuild or re-onboard guidance.
If that bounded retry is exhausted, or if gateway restart reports SUPERVISOR_BUSY, wait for the active request to finish and retry the command.
If the error mentions SUPERVISOR_NOT_RUNNING and trusted recreation could not proceed, SUPERVISOR_REBUILD_REQUIRED, a missing nemoclaw-gateway-control helper, or a missing managed controller, the sandbox image may predate the current lifecycle contract.
An exact SUPERVISOR_UNAVAILABLE result instead means the managed controller refused the current supervisor state rather than guessing which same-UID process is the gateway.
The current recovery action and any managed settle confirmation stop immediately.
If recover reports this result, follow its host-side gateway restart guidance.
If restart also reports SUPERVISOR_UNAVAILABLE, or the image is incompatible, rebuild the image:
For a custom image, update its Dockerfile to preserve the current NemoClaw entrypoint, root-only gateway control helper, root-only managed controller, and shared supervisor library before rebuilding.
OpenClaw startup reports OPENCLAW_LOCKED_PARENT_UNPROTECTED
This refusal means the OpenClaw config directory is root-owned but /sandbox is not root:sandbox 1775, so the sandbox identity could rename the entire .openclaw lock root.
PID 1 refuses startup before migration or config reads, and shields up cannot repair the parent after that refusal.
Do not change the parent ownership manually or accept the current bytes as a new seal.
If you have a trusted host-side snapshot from before the incident, record its selector, destroy the refused sandbox, re-onboard the same sandbox name from trusted host configuration, and restore that snapshot:
Destroying the refused sandbox permanently discards state newer than the selected snapshot, so confirm that the host-side snapshot exists before destruction. Without a trusted pre-incident snapshot, recreate the sandbox from host-side onboarding configuration only if you accept losing the inaccessible in-sandbox state.
Sandbox is running an outdated agent version
After upgrading NemoClaw, nemoclaw <name> connect and nemoclaw <name> status warn if the sandbox is running an older agent version than the current image.
To upgrade the sandbox while preserving workspace state, run:
The rebuild command backs up state, destroys the old sandbox, recreates it with the current image, and restores state. Create a snapshot before rebuilding if you want an additional safety net:
Sandbox shows as stopped
When status reports sandbox_container_stopped, Docker still has a container for the sandbox, but the container is not running.
Use the lightest recovery path first instead of rebuilding immediately.
-
Confirm Docker can still see the labeled container.
If a container is listed, start it:
-
Run status recovery from the host.
On Docker-driver hosts, status also attempts non-destructive recovery when OpenShell reports the sandbox as missing but Docker still has a stopped
openshell.ai/sandbox-name=<name>container or the latest GPU-backup sibling. A successful recovery prints that the sandbox was recovered from Docker and then shows the refreshed OpenShell state. -
If the sandbox is running but the agent gateway or dashboard forward is still down, recover the in-sandbox gateway and forwards:
-
Rebuild only if the sandbox cannot be restarted or status still cannot recover it while the local registry entry exists:
Rebuild recreates the sandbox from recorded metadata and preserves supported workspace and agent state. If the sandbox was intentionally deleted and you want a clean setup, run
nemoclaw <name> destroyto remove the stale local entry, then runnemoclaw onboard.
Sandbox is registered locally but missing from the gateway
After a gateway restart, host reboot, or manual OpenShell cleanup, NemoClaw may still have a local registry entry for a sandbox that the live gateway no longer lists.
nemoclaw <name> status and nemoclaw <name> connect preserve that local registry entry and print recovery guidance instead of deleting it automatically.
Run nemoclaw <name> rebuild --yes when you want NemoClaw to recreate the sandbox from the recorded metadata, or run nemoclaw <name> destroy when you intentionally want to remove the stale entry.
Status shows “not running” inside the sandbox
This is expected behavior. When checking status inside an active sandbox, host-side sandbox state and inference configuration are not inspectable. The status command detects the sandbox context and reports “active (inside sandbox)” instead.
Run openshell sandbox list on the host to check the underlying sandbox state.
Git clone fails with a certificate verification error
In networks that inspect TLS, OpenShell injects a proxy CA bundle into the sandbox.
Current NemoClaw exports that bundle as GIT_SSL_CAINFO during sandbox startup and persists it for nemoclaw <name> connect sessions, so Git can trust the proxy CA.
It also forwards standard CA bundle variables for subprocesses, including GIT_SSL_CAPATH, CURL_CA_BUNDLE, and REQUESTS_CA_BUNDLE.
If Git still reports server certificate verification failed, reconnect to the sandbox and check that the CA variables are present:
grep exits non-zero when it finds no matches, so empty output (with the trailing || true) simply means none of these CA variables are set in the current shell.
If they are missing on an older sandbox, upgrade NemoClaw and run:
External channel TLS fails behind a corporate MITM proxy (NET:FAIL)
On networks where a corporate proxy re-signs external TLS, endpoints such as api.telegram.org can fail certificate verification even when network policy allows the connection.
Logs show the request opening as NET:OPEN ... api.telegram.org:443 followed by NET:FAIL because the corporate root is not in the OpenShell trust path.
Provide the corporate CA before onboarding, then onboard or rebuild the sandbox.
Refer to Configure Corporate CA Trust for source precedence, host anchor discovery, image and runtime trust, custom Dockerfile requirements, import validation, and NEMOCLAW_CORPORATE_CA_IMPORT=0.
If the import does not occur, check onboarding output for baking corporate proxy CA from ... or a warning that the selected source was skipped.
A request inside the sandbox fails with CONNECT tunnel failed, response 403
Sandbox outbound network access is denied by default and enforced by the OpenShell proxy. When a request targets a host that no applied policy preset allows, the proxy refuses the tunnel and tools surface only the protocol-level error:
This is a network-policy denial, not a tool or certificate problem.
When you run a command through nemoclaw <name> exec -- ... and it exits non-zero, NemoClaw checks the sandbox audit log for a policy denial recorded after the command started.
If it finds one, it appends a short breadcrumb to stderr after the tool’s own output, naming the denied host:port when it can be extracted safely and showing the commands below:
If the log probe fails, no breadcrumb is added. Unsafe endpoint data is omitted from the breadcrumb, and an invalid sandbox name is shown as <name>.
An IPv6 target is named in its RFC 3986 bracketed form:
The tool’s own stdout/stderr bytes and its exit code are left unchanged. The breadcrumb is printed by the host CLI after the command finishes, and only for a genuine failure with a fresh denial. A command that succeeds, or one that fails for an unrelated reason, prints no breadcrumb. Set NEMOCLAW_NO_POLICY_HINT to any non-empty value other than 0 or case-insensitive false (for example, 1, true, TRUE, yes, or YES) to suppress it entirely.
The first interactive nemoclaw <name> connect shell also prints a one-line reminder of this denial signature and the logs command below.
The reminder is shown once per top-level interactive session, and only when all of these hold: an egress proxy is configured, the shell is interactive with a terminal attached to stderr, and it is a top-level shell (not a nested subshell or pane).
Suppress it with NEMOCLAW_NO_POLICY_HINT=1.
On OpenShell 0.0.44 or newer the reminder names your real sandbox; on older OpenShell it shows <name> as a placeholder — run nemoclaw list to see your sandbox names.
If the reported sandbox name contains characters that are not valid in a sandbox name (uppercase letters, underscores, control characters, and similar) or exceeds 63 characters, the reminder shows the <name> placeholder for safety rather than echoing the untrusted value.
The reminder is intentionally proactive: the denial itself is surfaced by the OpenShell proxy, so the curl/git error text is left unchanged and the reminder points you to the logs instead.
To see which rule denied the request, read the merged logs from the host:
If the host should be reachable, allow it with a preset or a custom preset:
Replace <preset> with a real preset name such as github, pypi, or npm. Run nemoclaw <name> policy-add with no preset to list the available presets.
Sandbox creation reports a TLS certificate mismatch
If sandbox creation reports a TLS or certificate mismatch, the OpenShell gateway certificate may have changed since the CLI last registered it. Remove the stale local gateway registration and then resume onboarding so NemoClaw refreshes the registration:
openclaw update hangs or times out inside the sandbox
This is expected for the current NemoClaw deployment model.
NemoClaw installs openclaw into the sandbox image at build time, so the CLI is image-pinned rather than updated in place inside a running sandbox.
Do not run openclaw update inside the sandbox.
Instead:
- Upgrade to a NemoClaw release that includes the newer
openclawversion. - If you build NemoClaw from source, bump the pinned
openclawversion inDockerfile.baseand rebuild the sandbox base image. - Run
nemoclaw <name> rebuildto recreate the sandbox with the updated image. The rebuild command automatically backs up workspace state before destroying the old sandbox and restores it afterward.
AWS EC2 Instance-Role Credential Discovery Is Unavailable
This is expected in an OpenClaw sandbox.
NemoClaw forces AWS_EC2_METADATA_DISABLED=true because OpenShell blocks the link-local EC2 Instance Metadata Service endpoint.
Existing sandboxes must use a current NemoClaw image before they receive this environment invariant.
Upgrade NemoClaw and rebuild the sandbox:
Reconnect and verify the value:
Expected output:
Only EC2 instance-role discovery is disabled.
Static access keys, bearer tokens, shared profiles, SSO and process credentials, web identity, and ECS container credentials remain eligible.
NemoClaw’s host-local Amazon Bedrock adapter is outside the sandbox credential-discovery boundary and remains available.
Do not add 169.254.169.254 to a network policy or override the variable.
Inference requests time out
Verify that the inference provider endpoint is reachable from the host. Check the active provider and endpoint:
The main Inference line probes https://inference.local/v1/models from inside the sandbox, so it reflects the route the agent actually uses.
If that line shows unhealthy, unreachable, or not probed, inspect the labeled diagnostic lines to identify the failing hop.
For local Ollama and local vLLM, Inference (ollama backend) or the corresponding local-backend line reports the host-side service separately.
For Local Ollama, current releases can also print Inference (auth proxy) when a proxy token is available.
If a local backend or auth-proxy diagnostic fails, start the backend or re-run onboarding so NemoClaw can recreate the proxy token, restart the proxy, and refresh the route.
For Ollama-backed OpenClaw sandboxes, agent passthrough uses the registered host route to warm an unloaded model after an Ollama daemon restart.
If that bounded warm-up fails or times out, NemoClaw reports the result and continues so OpenClaw can emit its canonical backend error.
If the endpoint is correct but requests still fail, check for network policy rules that may block the connection. Then verify the credential and base URL for the provider you selected during onboarding.
If you entered an AWS Bedrock Runtime URL such as https://bedrock-runtime.us-east-1.amazonaws.com in the Other Anthropic-compatible endpoint flow, NemoClaw auto-detects it and routes sandbox traffic through a host-local adapter.
Use the raw Bedrock Runtime host, not an Anthropic /v1/messages path, and verify that the model ID or inference profile ID is valid for that region.
For auth, export AWS_BEARER_TOKEN_BEDROCK, AWS_PROFILE, or standard IAM environment credentials before onboarding; if you paste a key at the COMPATIBLE_ANTHROPIC_API_KEY prompt, NemoClaw uses it only as the adapter’s Bedrock bearer token.
Region errors usually mean the pasted endpoint region, AWS_REGION, AWS_DEFAULT_REGION, or the model/inference profile ID do not match.
For Ollama, vLLM, NIM, and compatible-endpoint inference validation, the default timeout is 180 seconds.
The managed NIM startup health wait uses a separate 15-minute (900-second) default and still exits early if the container stops before it becomes healthy.
On Docker 29.x or hosts using the containerd image store, managed NIM onboarding resolves and pulls the host-platform image digest when NGC exposes a multi-architecture image index.
If you still see NGC repository-format or attestation errors, confirm Docker can run docker manifest inspect for the selected image and that you are logged in to nvcr.io.
If large prompts still cause timeouts, increase it with NEMOCLAW_LOCAL_INFERENCE_TIMEOUT before re-running onboard:
For local Ollama and vLLM, onboarding retries the container reachability check and can fall back to the host-side health check when the local backend is healthy. If Ollama times out during a cold model load, NemoClaw retries once with a 300-second probe budget before failing. If all attempts fail, the error includes container reachability diagnostics such as HTTP status and host gateway resolution.
NEMOCLAW_LOCAL_INFERENCE_TIMEOUT only covers the inference-server validation probe.
The post-create readiness wait has its own budget (NEMOCLAW_SANDBOX_READY_TIMEOUT); refer to Sandbox onboard times out with “did not become ready within Ns” for the readiness path.
Sandbox onboard times out with “did not become ready within Ns”
Onboarding ends with:
This is a separate budget from NEMOCLAW_LOCAL_INFERENCE_TIMEOUT.
It covers the readiness wait that follows sandbox creation, including in-sandbox boot, OpenClaw start, and policy load.
It does not cover the inference probe.
The 180-second default fits typical workstations but can be exceeded when:
- The host is building or uploading the sandbox image for the first time (cold caches, slow link).
- The selected model is large (70B+ parameters or 4-bit/8-bit quantisations that take time to memory-map).
- Onboarding runs on a remote VM where image upload to the gateway streams over the network (for example DGX Station first-run installer).
Raise the budget before re-running onboard:
The variable accepts seconds and applies to the readiness wait only.
When the deadline expires, NemoClaw deletes the partially-created sandbox before printing the retry hint, so the next nemoclaw onboard starts from a clean state.
If readiness still fails after the extended budget, inspect the gateway and sandbox status:
Sandbox onboard fails with “entered Error phase before it became ready”
Onboarding ends with:
On a fresh onboard the OpenShell gateway can (re)start its supervisor session and re-register the just-created sandbox.
During that window openshell sandbox list briefly reports the sandbox in the transient Error phase before it flips to Ready, as seen on DGX Spark when supervisor restart races the sandbox bootstrap.
NemoClaw tolerates a bounded run of consecutive Error polls (default 30 polls / ~60s) so this transient recovers on its own; only Error that persists past the debounce window is treated as terminal. Failed and CrashLoopBackOff are always terminal and fail immediately.
If your host needs a longer window (slower re-registration), raise the debounce; to fail fast on the first Error poll, set it to 1:
If the failure persists after the debounce, the sandbox is genuinely stuck — inspect the retained diagnostics and gateway state:
Agent fails at runtime after onboarding succeeds with a compatible endpoint
Some OpenAI-compatible servers (such as SGLang) expose /v1/responses but their
streaming mode is incomplete.
OpenClaw requires granular streaming events like response.output_text.delta
that these backends do not emit.
For the compatible-endpoint provider, NemoClaw now defaults to
/v1/chat/completions and skips the Responses API probe entirely unless you
opt in.
If you onboarded an older release that selected /v1/responses, re-run
onboarding so the wizard rebuilds the image with chat completions:
If you previously set NEMOCLAW_PREFERRED_API=openai-responses to force the
Responses API, unset it before re-running onboard.
When you enable Telegram messaging with an OpenAI-compatible endpoint, onboarding also checks inference.local from inside the sandbox.
If that smoke check fails, fix the compatible-endpoint base URL, credentials, model, or network route before testing the Telegram bot again.
Do not rely on NEMOCLAW_INFERENCE_API_OVERRIDE alone.
It patches the config at container startup but does not update the Dockerfile ARG baked into the image.
A fresh nemoclaw onboard is the reliable fix.
Tool calls appear as assistant text
Local model servers must return structured tool_calls for OpenClaw to dispatch a tool.
When the inference response contains only text that resembles a tool request, the gateway treats it as ordinary assistant text and no tool runs.
The TUI can display a response such as:
This symptom is different from a network or policy block.
nemoclaw <name> status, nemoclaw <name> logs, and nemoclaw debug --quick can all look healthy while conversation-level tool dispatch fails.
Ollama can serve local chat and some simple tool surfaces, but agent loops with several tools, long instructions, or multi-turn dispatch need a server that returns structured tool calls consistently.
For persistent agent use, set up vLLM with automatic tool choice and the tool-call parser that matches the model family.
After the parser-aware server is ready, re-run onboarding.
For a server detected on localhost:8000, select the Local vLLM entry marked running (suggested); for another address, select Other OpenAI-compatible endpoint.
On generic hosts, the Local vLLM entry includes an experimental label; on DGX Spark or DGX Station, it does not.
Do not rely on direct edits to openclaw.json for a persistent provider change.
NemoClaw-managed rebuilds can overwrite those edits, while onboarding keeps the sandbox image, OpenShell route, and host-managed credentials aligned.
Ask the agent to perform an action that requires a tool, then confirm that the TUI does not show a JSON blob as assistant text, the gateway log shows tool dispatch followed by an answer, and nemoclaw <name> status reports the intended local vLLM or compatible provider.
If JSON still appears as text, confirm that vLLM started with automatic tool choice and the parser required by the model family.
Onboarding fails with duplicate Anthropic message_start events
Validation for an OpenClaw Other Anthropic-compatible endpoint selection ends with an error like:
For OpenClaw custom Anthropic routes, NemoClaw sends a stream: true request to /v1/messages and validates the SSE event sequence (exactly one message_start, at least one content_block_delta, and a message_stop).
This error means the streaming layer on the endpoint or gateway is malformed even though its non-streaming responses are valid.
A working non-streaming response does not imply that streaming works.
Some inference gateways proxy plain requests correctly but corrupt the SSE stream, for example by emitting message_start twice for one request.
OpenClaw uses the streaming path, so without this check the defect would first surface inside the sandbox as a runtime failure.
Hermes and OpenAI-compatible-only agents use the endpoint’s /v1/chat/completions surface for custom Anthropic selections instead.
Current onboarding validates that surface and does not reject those agents because of a malformed native /v1/messages stream they will not use.
An older Hermes sandbox that still uses native Anthropic Messages can report that no final response was produced; re-run onboarding to select and validate the managed Chat Completions route.
Fix the streaming layer on the endpoint or gateway, or onboard with a different Anthropic-compatible endpoint.
The official Anthropic provider does not run this check and is not affected.
If an OpenClaw sandbox created by an older release fails at runtime with an empty final response on an Anthropic-compatible endpoint, re-run nemoclaw onboard so the streaming check can diagnose the endpoint.
If the endpoint serves a reasoning-only model, set NEMOCLAW_REASONING=true to skip the streaming check.
Streaming defects then surface at runtime instead of during onboarding.
NEMOCLAW_DISABLE_DEVICE_AUTH=1 does not change an existing sandbox
This is expected behavior.
NEMOCLAW_DISABLE_DEVICE_AUTH is a build-time setting used when NemoClaw creates the sandbox image.
Changing or exporting it later does not rewrite the baked openclaw.json inside an existing sandbox.
If you need a different device-auth setting, rerun onboarding so NemoClaw rebuilds the sandbox image with the desired configuration. For the security trade-offs, refer to Security Best Practices.
openclaw.json is empty after changing inference
Some runtime inference changes can leave /sandbox/.openclaw/openclaw.json empty if the write fails partway through.
When that happens, OpenClaw commands may report that the config is empty instead of showing a raw JSON parse error.
Current NemoClaw sandboxes capture a known-good config baseline after a successful startup.
On the next sandbox startup, NemoClaw restores openclaw.json from OpenClaw’s last-good copy when available, or from the NemoClaw baseline.
Recovery validates the config tree and selected source before atomically replacing openclaw.json and .config-hash.
If it detects an unsafe link, unexpected owner, or path change, startup fails closed without following or modifying the unsafe target.
If the sandbox still cannot start or reports that no baseline is available, rebuild it from the host:
openclaw channels add or remove is blocked inside the sandbox
This is expected.
The messaging channel list is frozen into the sandbox’s container image when the image is built during nemoclaw onboard or nemoclaw rebuild (the selected channel names are passed to the docker build as NEMOCLAW_MESSAGING_CHANNELS_B64 and written into agent config such as /sandbox/.openclaw/openclaw.json for OpenClaw or /sandbox/.hermes/.env for Hermes).
Changes made inside the running sandbox do not persist across rebuilds, so openclaw channels commands that mutate the config are intercepted.
NemoClaw’s sandbox entrypoint installs a guard that intercepts openclaw channels <add|remove> and prints an actionable error pointing at the host-side commands below, instead of letting the call fail deep in the binary with a raw EACCES trace.
Run the equivalent host-side command instead:
channels add registers credentials with the OpenShell gateway and channels remove clears them.
Both offer to rebuild the sandbox so the image reflects the new channel set.
In non-interactive mode (NEMOCLAW_NON_INTERACTIVE=1), the commands stage the change and leave the rebuild to a follow-up nemoclaw <sandbox> rebuild.
WeChat and WhatsApp are experimental.
Review Choose Messaging Channels before enabling them.
WeChat captures its bot token through a host-side QR scan during nemoclaw onboard or channels add wechat.
You scan the iLink QR from WeChat on your phone and NemoClaw registers the captured token with the OpenShell gateway.
WhatsApp pairs entirely inside the sandbox.
NemoClaw advertises WhatsApp for OpenClaw and Hermes sandboxes after you add the channel on the host.
Run openclaw channels login --channel whatsapp inside OpenClaw sandboxes, or run hermes whatsapp inside Hermes sandboxes.
scripts/rcf_patch.py is missing from the blueprint
scripts/rcf_patch.py is intentionally absent from current NemoClaw blueprints.
Older QA plans used that helper for a Dockerfile “Patch-4” test that corrupted the build-time replaceConfigFile monkey-patch and expected ERROR: Patch 4 (replaceConfigFile EACCES) not applied.
The old Patch-4 fail-closed test no longer applies because NemoClaw no longer patches OpenClaw’s compiled replaceConfigFile source at image build time.
Current sandboxes use a mutable-default config model instead.
Before a reviewed host-side lockdown, /sandbox/.openclaw/openclaw.json is group-writable by the sandbox and gateway users, so OpenClaw config mutations should write normally rather than requiring an EACCES swallow.
After lockdown, runtime config mutations should fail cleanly or route users to the supported host-side NemoClaw command.
To validate this area now, use the config lifecycle tests instead of looking for rcf_patch.py:
openclaw config set or unset is blocked inside the sandbox
This is expected.
NemoClaw builds the sandbox’s OpenClaw configuration (/sandbox/.openclaw/openclaw.json) from host-side onboarding, rebuild, inference, policy, and messaging inputs.
Fresh sandboxes keep that file writable by default so the agent can manage runtime state, but direct in-sandbox edits are not the supported or durable path for NemoClaw-managed settings.
NemoClaw’s sandbox entrypoint installs a guard that intercepts openclaw config set and openclaw config unset and prints an actionable error, because changes made inside the running sandbox do not persist across rebuilds.
For most configuration changes, exit the sandbox and rerun onboarding:
If NemoClaw reports a resumable failed onboarding session, run nemoclaw onboard --resume instead.
This rebuilds the sandbox with your updated settings.
For advanced live edits, use the host-side config command instead of running openclaw config set inside the sandbox:
Host-side config set validates any HTTP or HTTPS URLs in the new value, including URLs nested inside JSON objects or arrays. NemoClaw rejects loopback, private, reserved, and internal hosts; DNS names must resolve successfully and must not resolve to private/internal addresses. HTTP URLs are written with the validated IP address pinned to reduce DNS-rebinding risk. Avoid putting credentials in config values; rotate provider credentials with the credential-management commands instead.
openclaw doctor --fix cannot repair Discord channel config inside the sandbox
This is expected in NemoClaw-managed sandboxes.
NemoClaw bakes channel entries into /sandbox/.openclaw/openclaw.json at image build time.
As a result, commands that try to rewrite the baked config from inside the sandbox, including openclaw doctor --fix, cannot repair Discord, Telegram, or Slack channel entries in place.
If your Discord channel config is wrong, rerun onboarding so NemoClaw rebuilds the sandbox image with the correct messaging selection.
Do not treat a failed doctor --fix run as proof that the Discord gateway path itself is broken.
If openclaw doctor reports that it moved Telegram single-account values under channels.telegram.accounts.default, rerun onboarding and rebuild the sandbox rather than trying to patch openclaw.json in place.
Current NemoClaw rebuilds bake Telegram in the account-based layout and set Telegram group chats to groupPolicy: open, which avoids the empty groupAllowFrom warning path for default group-chat access.
openclaw doctor --fix tightened config permissions and the gateway can no longer save config
In a mutable NemoClaw sandbox, the gateway UID and the sandbox UID share the sandbox group, so /sandbox/.openclaw is setgid and group-writable (2770) and openclaw.json is group-writable (660).
OpenClaw’s openclaw doctor --fix enforces its own single-user 700/600 layout, so running it inside the sandbox strips group write and breaks gateway-side config writes (for example, control-UI toggles that mutate openclaw.json).
When you invoke the command through the host-side one-shot path, NemoClaw restores the mutable contract as the command exits:
When cleanup succeeds, exec preserves the openclaw doctor --fix exit code.
If cleanup cannot inspect, restore, or verify the mutable config permission contract, it returns a cleanup failure instead and prints OpenClaw permission cleanup failed (...) with the command and cleanup statuses.
Do not recursively change ownership or permissions over a config tree that failed this safety check.
Inspect the reported condition and rebuild from trusted host-side configuration if the tree or image boundary is not the expected NemoClaw layout.
If you ran openclaw doctor --fix from an interactive connect shell or use an older sandbox image, repair the mutable contract without rebuilding:
nemoclaw <sandbox> doctor reports the drift as a Config permissions warning, and --fix restores 2770/660.
Restarting the sandbox repairs the same drift automatically when the config tree passes its safety checks, and NemoClaw’s own rebuild re-applies the contract after its post-upgrade openclaw doctor --fix step.
For a persisted root-owned 700/600 tree, startup reclaims ownership only when both fixed config files have that exact posture under the expected sandbox-owned parent.
Other root-owned layouts, links, mounts, and ambiguous metadata fail closed so startup cannot mistake a shields-locked or unsafe tree for mutable drift.
If startup reports [SECURITY] Refusing mutable config permission normalization, NemoClaw stops startup without following or modifying the unsafe target; safe permission repairs completed before detection are not rolled back.
Rebuild with the current image and trusted host-side configuration instead of repairing the tree recursively.
When verifying gateway write access by hand, step down to the gateway UID with the image’s installed mechanism so the sandbox group membership is initialized:
Do not probe with su -s /bin/sh gateway ...: su does not initialize the gateway’s supplementary groups the same way, so a group-write probe can spuriously report EACCES even when the mutable contract is intact.
A NemoClaw sandbox has two intentional permission states for /sandbox/.openclaw; 700/600 is not one of them:
- Mutable default:
/sandbox/.openclawis2770 sandbox:sandboxandopenclaw.jsonis660 sandbox:sandbox. Both the sandbox user and the gateway (samesandboxgroup, different UID) can write config, so control-UI toggles persist. - Host-locked state:
openclaw.jsonis read-only for in-sandbox writers and the config dir is owned byroot, with the immutable bit set where available. No in-sandbox writes are expected; use the host-sidenemoclaw <sandbox> config setflow described inopenclaw config setfails with a permission error on Brev. 700/600(drift): the layout that upstreamopenclaw doctor --fiximposes inside a mutable sandbox. It is not a supported NemoClaw state; recover withnemoclaw <sandbox> doctor --fixor a sandbox restart.
Discord bot logs in, but the channel still does not work
Separate the problem into two parts:
-
Baked config and provider wiring
Check that onboarding selected Discord and that the sandbox was created with the Discord messaging provider attached. If Discord was skipped during onboarding, rerun onboarding and select Discord again.
-
Native Discord gateway path
Successful login alone does not prove that Discord works end to end. Discord also needs a working gateway connection to
gateway.discord.gg. If logs show errors such asgetaddrinfo EAI_AGAIN gateway.discord.gg, repeated reconnect loops, or a400response while probing the gateway path, the problem is usually in the native gateway/proxy path rather than in the baked config.
Common signs of a native gateway-path failure:
- REST calls to
discord.comsucceed, but the Discord channel never becomes healthy gateway.discord.ggfails with DNS resolution errors- the WebSocket path returns
400instead of opening a tunnel - native command deployment fails even though the bot token itself is valid
In that case:
- keep the Discord policy preset applied
- verify the sandbox was created with the Discord provider attached
- inspect gateway logs and blocked requests with
openshell term - treat the failure as a native Discord gateway problem, not as a bridge startup problem
Discord preset validation behind a proxy
The built-in Discord policy preset intentionally allows the Node binaries used by the messaging runtime and does not allow curl.
As a result, curl -s https://discord.com failing, hanging, or printing no output is not proof that the Discord preset is broken.
Behind the OpenShell proxy, direct DNS-only checks can also be the wrong signal.
For example, dns.resolve("gateway.discord.gg") can fail even when HTTPS requests routed through the proxy are healthy.
Use Node HTTPS as the manual REST probe:
To check Discord CDN egress, use the same Node HTTPS path:
Any HTTP status from these probes means the Node process reached the endpoint; the exact status can vary by unauthenticated path.
If the Node REST probe works but the Discord channel is still unhealthy, investigate the native gateway path instead of widening the preset.
Check the gateway logs and blocked-request output with openshell term, and look for gateway.discord.gg connection or WebSocket upgrade failures.
Messaging bridge appears running but no messages arrive
Bot tokens for Telegram (getUpdates), Discord (gateway), and Slack (Socket Mode) only allow one active consumer per token. If two NemoClaw sandboxes are configured with the same bot token, each one kicks the other off its polling connection and neither delivers messages. nemoclaw status still reports the bridge as running because the gateway process itself is alive.
For Telegram group chats, first check BotFather privacy mode.
New Telegram bots default to privacy mode enabled, which prevents group messages from reaching getUpdates even when the user mentions the bot.
In @BotFather, run /setprivacy, choose the bot, and choose Disable.
Then remove the bot from the affected group and add it back; Telegram applies the privacy-mode change to group delivery only after the bot rejoins.
For Telegram direct messages, make sure the rebuilt sandbox has a DM allowlist.
Set TELEGRAM_ALLOWED_IDS before rebuild; TELEGRAM_AUTHORIZED_CHAT_IDS and TELEGRAM_CHAT_ID are accepted as compatibility aliases.
Keep the aliases until QA automation and public repro templates have stopped exporting them for at least one full release.
Bot API sendMessage sends from the bot to a chat, so it only proves outbound Telegram API access.
To prove inbound agent routing, send a message from the Telegram client as an allowed user and then watch the gateway log for the agent turn and outbound reply.
For a reproducible outbound runtime check, run NEMOCLAW_RUN_LIVE_E2E=1 npx vitest run --project e2e-live test/e2e/live/messaging-providers.test.ts --silent=false --reporter=default with NVIDIA_INFERENCE_API_KEY set.
The check imports the installed OpenClaw Telegram runtime-api.js, calls sendMessageTelegram through an OpenShell-rewritten credential against a host-side fake Telegram API, and verifies the captured chat, text, token rewrite, and absence of unresolved placeholders.
When TELEGRAM_BOT_TOKEN_REAL and TELEGRAM_CHAT_ID_E2E are also set, the same lane performs an additional real outbound send; it does not prompt for or claim an interactive inbound reply.
To diagnose, open a shell in the sandbox and inspect the gateway log:
A repeating line like the following confirms the conflict:
To fix, run nemoclaw <other-sandbox> destroy on whichever sandbox should stop polling, or rerun onboarding on it with the channel disabled. Current NemoClaw warns at nemoclaw onboard time when another sandbox already has the same channel enabled, but sandboxes created before that check was added may still be in a conflict loop.
Landlock filesystem restrictions silently degraded
After sandbox creation, NemoClaw checks whether the host kernel supports Landlock (Linux 5.13+). If the kernel is too old or you are running on macOS (where the Docker VM kernel may lack Landlock), a warning prints:
This warning is informational and does not block sandbox creation. The sandbox runs without kernel-level filesystem restrictions, relying on container mount configuration instead. For full filesystem enforcement, run on a Linux kernel 5.13 or later (Ubuntu 22.04 LTS and later include Landlock support).
Sandbox lost after gateway restart
Sandboxes created with OpenShell versions older than 0.0.24 can become unreachable after a gateway restart because SSH secrets were not persisted.
Running nemoclaw onboard automatically upgrades OpenShell to 0.0.24 or later during the preflight check.
After the upgrade, recreate the sandbox with nemoclaw onboard.
DNS-backed HTTPS endpoint is not supported
NemoClaw rejects an explicit custom endpoint when it resolves a public HTTPS hostname but cannot pin the same peer address across the downstream OpenShell runtime boundary while preserving TLS SNI and host validation.
This can appear during a direct blueprint run, custom-endpoint onboarding, or a host-side config set write.
It can also appear during a runtime nemoclaw inference set switch.
Use an HTTPS IP-literal endpoint whose certificate is valid for that address. If your deployment permits non-TLS provider traffic, you can instead use a public HTTP endpoint that NemoClaw can rewrite to a DNS-pinned address. Do not bypass the check with a private or internal address or by editing the persisted sandbox config directly. For the full endpoint rules, refer to Meet Custom Endpoint Security Requirements.
Agent cannot reach external hosts through a proxy
NemoClaw uses a default proxy address of 10.200.0.1:3128 (the OpenShell-injected gateway).
If your environment uses a different proxy, set NEMOCLAW_PROXY_HOST and NEMOCLAW_PROXY_PORT before onboarding:
These are build-time settings baked into the sandbox image.
Changing them after onboarding requires re-running nemoclaw onboard to rebuild the image.
When HTTP_PROXY or HTTPS_PROXY is set on the host, NemoClaw adds localhost, 127.0.0.1, ::1, 0.0.0.0, the container-host aliases host.docker.internal and host.containers.internal, and the managed inference hostname inference.local to NO_PROXY for host-side subprocesses and for the env forwarded into openshell sandbox create.
This keeps local Ollama health checks, model pulls, and managed inference traffic from being chained through a corporate or desktop proxy at the sandbox-create boundary, while preserving the proxy for external hosts.
Inside the running sandbox, processes continue to use the OpenShell L7 proxy for inference.local so OpenShell’s internal routing, DNS, and audit boundaries stay intact.
Agent cannot reach a host-side HTTP service
When a sandbox needs to call an HTTP service running on the host, use the normal OpenShell network policy path.
Expose the service on a host IP address that the OpenShell gateway can reach, create a custom NemoClaw policy preset for that IP and port, and apply it with nemoclaw <sandbox> policy-add --from-file.
The sandbox request then flows through the OpenShell proxy while NemoClaw preserves the existing live policy entries.
Do not rely on host.docker.internal or host.openshell.internal as a general-purpose host-service path.
Those names may appear in the sandbox’s /etc/hosts, but in OpenShell’s sandbox network they are not guaranteed to point at a reachable host gateway.
Bypassing the proxy with --noproxy '*' also bypasses network policy enforcement and audit.
First, make sure the host-side service listens on a non-loopback address.
For example, a health endpoint on port 50001 should be reachable from the host IP, not only from 127.0.0.1:
Expected output:
Then create a custom NemoClaw preset for the host-side service.
Replace 10.0.0.5, 50001, paths, methods, and binaries with the service you want the sandbox to reach:
Apply the preset to the running sandbox with the NemoClaw CLI:
After you apply the policy, retry the request from inside the sandbox without disabling the proxy:
Expected output:
If the request is still denied, check the blocked request in openshell term.
The policy binaries list must include the executable path that actually made the request.
If the response changes from policy_denied to upstream_unreachable, the policy matched, but the OpenShell gateway could not reach the host IP and port.
Agent cannot reach an external host
OpenShell blocks outbound connections to hosts not listed in the network policy. Open the TUI to see blocked requests and approve them:
To permanently allow an endpoint, add it to the network policy. Refer to Customize the Network Policy for details.
Dashboard not reachable after setting a custom port
If you ran nemoclaw onboard with a custom dashboard port and onboarding completed but the dashboard URL is unreachable, the sandbox was most likely created with an older NemoClaw version that did not pass the dashboard port into the sandbox at startup.
The browser may show connection refused or fail to load the page.
The gateway inside the sandbox continued listening on the default port 18789 while the SSH tunnel forwarded the custom port, leaving nothing at the other end of the tunnel.
Re-run onboarding on the current NemoClaw release with the desired port.
Current versions derive the dashboard port from CHAT_UI_URL automatically and inject it into the sandbox:
If you need to run multiple sandboxes at different ports at the same time, refer to Running multiple sandboxes simultaneously.
Control UI config endpoint returns 404 or non-JSON
The Control UI loads its runtime configuration from a gateway endpoint, not from a static
controlui.bootstrap.config.json file. No controlui.bootstrap.config.json path is served,
so requesting it returns HTTP 404 Not Found with a short plain-text body, and piping that
response to jq fails with a parse error such as Invalid numeric literal.
The supported Control UI config endpoint is /__openclaw/control-ui-config.json, served by
the OpenClaw gateway on the forwarded dashboard port. It is gated by the gateway auth token:
- An unauthenticated request returns
HTTP 401 Unauthorizedwith a JSON body ({"error":{"message":"Unauthorized","type":"unauthorized"}}), which is already valid JSON. - An authenticated request returns
HTTP 200 OKwith the Control UI config as JSON.
Resolve the forwarded dashboard port, then authenticate with the gateway token from
nemoclaw <name> gateway-token:
The token is sensitive; treat it like a password and do not log, share, or commit it. For
browser access, use the tokenized URL from nemoclaw <name> dashboard-url instead of
calling the config endpoint directly.
Ollama auth proxy did not start
NemoClaw keeps Ollama bound to 127.0.0.1:11434 and starts a token-gated
reverse proxy on 0.0.0.0:11435 so the sandbox can reach Ollama without
exposing it to the local network.
If the proxy fails to start, onboarding exits before configuring inference.
Check whether the proxy port is occupied by another process:
Stop the conflicting process and re-run nemoclaw onboard.
The wizard cleans up stale proxy processes from previous runs automatically,
so most failures resolve by retrying.
The proxy token is persisted to ~/.nemoclaw/ollama-proxy-token with 0600
permissions.
If the file is missing or unreadable after a host reboot, re-running
nemoclaw onboard regenerates it.
Ollama auth proxy is unreachable from the sandbox
On native Linux Docker-driver hosts, a host firewall can allow the host proxy check but block sandbox traffic to the Ollama auth proxy. When that happens, onboarding exits before it saves the inference route and prints output like:
Apply the ufw command printed by onboarding, then rerun onboarding.
If the message does not include a subnet, derive it from the OpenShell Docker network:
Docker Desktop, WSL, and hosts without the OpenShell Docker network use different routing models. In those cases NemoClaw treats an unavailable sandbox-side probe as non-blocking and relies on the regular proxy health check.
host.docker.internal does not reliably reach the host from the sandbox
Configuring an inference provider with a base URL like http://host.docker.internal:11434/v1 does not reliably reach a host Ollama service from inside the OpenShell sandbox.
OpenShell runs sandboxes inside a k3s network, where host.docker.internal is not a portable host-service route.
Depending on the platform, it may fail DNS resolution or resolve to an internal gateway/bridge address where the host’s port 11434 is not forwarded.
The sandbox then sees a DNS failure or connection refused:
Expected output:
Expected output:
For local Ollama, use the auth-proxy URL that NemoClaw’s “Local Ollama” onboard option configures automatically:
host.openshell.internal resolves to the same gateway IP, and the
token-gated Ollama auth proxy binds port
11435 there and forwards requests to 127.0.0.1:11434 on the host.
If you need a different host service exposed to the sandbox, route it through
the OpenShell gateway rather than relying on host.docker.internal.
Refer to issue #3136.
Local inference health check resolves to IPv6
Local inference health checks now use 127.0.0.1 instead of localhost.
On systems where localhost resolves to ::1 first, older NemoClaw releases
could probe the wrong address and report the local backend as unreachable
even when it was running.
If you see this on a current NemoClaw release, verify that the local backend
binds an IPv4 address and not only ::1.
Blueprint run failed
View the error output for the failed blueprint run:
Use --follow to stream logs in real time while debugging.
DGX Spark
For an end-to-end walkthrough with local inference on DGX Spark, refer to the NVIDIA Spark playbook.
Host freezes or logs NVRM NV_ERR_NO_MEMORY under local vLLM load
Treat a full host freeze separately from an agent tool-call hang.
If the Spark stops responding to SSH and ping, and the journal contains NVRM NV_ERR_NO_MEMORY or no software-side crash record, first isolate the local inference server before changing MCP or network policy configuration.
For onboarding-time context, refer to Use an Existing Server.
Check whether vLLM is a bring-your-own server or the NemoClaw managed Spark profile:
For an existing vLLM server, inspect its launch arguments:
Large checkpoints without explicit quantization, very long --max-model-len values, high --gpu-memory-utilization, and multiple concurrent sequences all consume the Spark’s shared CPU/GPU memory pool.
Before reintroducing agent tools, restart vLLM with a smaller envelope, for example:
If the host still logs NVRM NV_ERR_NO_MEMORY while loading the model, switch to a smaller or quantized checkpoint.
For managed setup, prefer NEMOCLAW_PROVIDER=install-vllm, which selects the Spark profile and its registered model-specific serve arguments.
After standalone vLLM is stable, re-run onboarding and add MCP servers back one group at a time.
CoreDNS CrashLoop after onboarding
If CoreDNS in the embedded k3s cluster crashes shortly after setup, it is usually because it resolves against 127.0.0.11, which does not route inside the gateway container.
Run fix-coredns.sh to point CoreDNS at the container gateway IP instead, then recreate the sandbox.
k3s cannot find a freshly built image
After building a new sandbox image, k3s inside the gateway container sometimes fails to pull it even though the image exists on the host.
Remove the gateway registration, stop any leftover host gateway process, then re-run setup.
GPU passthrough on Spark
GPU passthrough is not CI-tested on DGX Spark.
It is expected to work when you pass --gpu and the NVIDIA Container Toolkit is configured.
Verify the toolkit is configured by running docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi from the host.
If nvidia-smi works on the host but onboarding says GPU passthrough was not enabled, install or repair the NVIDIA Container Toolkit, then run sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker.
If a reusable gateway was previously started without GPU passthrough, NemoClaw replaces it automatically only when no other registered sandboxes depend on it, or when --recreate-sandbox is recreating the only registered sandbox with the same name.
When shared gateway cleanup would be unsafe, follow the targeted destroy or gateway-removal commands printed by onboarding.
unresolvable CDI devices nvidia.com/gpu=all during gateway start
Recent NVIDIA Container Toolkit installs configure the Docker daemon for Container Device Interface (CDI) device injection, which OpenShell’s gateway start --gpu then auto-selects.
If no nvidia.com/gpu CDI spec has been generated on the host yet, gateway start fails with Docker responded with status code 500: CDI device injection failed: unresolvable CDI devices nvidia.com/gpu=all.
Outside Station Express, the standard NemoClaw installer detects this gap before onboarding, first tries to enable the NVIDIA CDI refresh systemd units, and can fall back to generating the spec directly with nvidia-ctk.
Station Express requires the packaged refresh lifecycle to work; if it fails or omits nvidia.com/gpu=all, inspect nvidia-cdi-refresh.service, repair it, and rerun the printed exact-commit install command.
If you run nemoclaw onboard directly, preflight prints the manual remediation instead.
The native Linux fix is the same on Docker hosts whose docker info advertises a non-empty CDISpecDirs.
On WSL with Docker Desktop, Docker may advertise CDI directories even though --device nvidia.com/gpu=all is not usable from the WSL distro.
For that runtime, NemoClaw skips Linux CDI repair and uses Docker’s --gpus compatibility path for sandbox GPU access.
This compatibility path can be retired once Docker Desktop exposes usable nvidia.com/gpu CDI specs inside WSL, or once OpenShell no longer requires host-visible CDI specs for Docker Desktop WSL GPU passthrough.
Enable the refresh units, verify they list nvidia.com/gpu entries, then rerun onboarding:
For other native Linux installations, if the refresh units are unavailable or do not generate CDI devices, generate the spec directly:
On WSL with Docker Desktop, confirm Docker Desktop WSL integration is enabled for your distro and verify Docker GPU access from WSL:
If GPU passthrough is not required on this host, rerun onboarding with --no-gpu instead.
GPU routing or compatibility patch failed
The route depends on the host environment and the operator control. Identify the matching path before applying the recovery guidance.
Ordinary native Linux bounded fallback
Ordinary Linux GPU onboarding uses native OpenShell GPU injection and stops on failure by default.
Unset, auto, and 0 all preserve this native-only confinement boundary.
NEMOCLAW_DOCKER_GPU_PATCH=fallback is the explicit operator authorization for one bounded retry.
With that control set, if sandbox creation rejects the native GPU flag before progress, the exact OpenShell-managed container labeled for that sandbox records a host runtime GPU-injection error, or an explicit nvidia-smi driver proof fails while that container’s immutable host configuration confirms that no GPU was attached, NemoClaw captures redacted diagnostics, deletes the incomplete sandbox, verifies that no OpenShell-managed Docker container labeled for that sandbox remains, and retries exactly once through the compatibility path.
Free-form build/list text and sandbox-reported CUDA output never independently authorize the broader retry.
Without corroborating host evidence, onboarding fails closed even when fallback is set and directs the operator to clean up and explicitly select compatibility with NEMOCLAW_DOCKER_GPU_PATCH=1 if desired.
Before the authorized retry, NemoClaw warns that the legacy GPU compatibility envelope recreates the OpenShell-managed Docker container and may relax container confinement compared with native injection.
Specifically, compatibility recreation adds SYS_PTRACE, adds apparmor=unconfined when the original container has no AppArmor option, and uses a compatibility policy that makes /proc writable for the NVIDIA runtime’s process-name initialization.
These broader settings are why onboarding warns before the swap and retains a native-only opt-out.
NemoClaw verifies cleanup with two stable checks (that sandbox is absent from the gateway list and no OpenShell-managed Docker containers labeled for that sandbox remain) before retrying through the compatibility path.
Cleanup is polled at most five times, one second apart, and both conditions must pass twice consecutively; otherwise onboarding stops before the retry.
These fail-closed safety limits are the internal constants STABLE_ABSENCE_CHECKS (2), MAX_CLEANUP_ATTEMPTS (5), and CLEANUP_POLL_INTERVAL_MS (1,000 ms); they are not configurable through environment variables.
The first observation is immediate, so the default bound performs at most four one-second sleeps plus the five gateway/container queries.
The bounds are intentionally fixed.
Allowing environment input to weaken or extend the cleanup proof would make a security gate deployment-dependent.
On a host that cannot prove absence within the bound, onboarding fails closed; select compatibility from the outset with NEMOCLAW_DOCKER_GPU_PATCH=1 instead of weakening the handoff proof.
If deletion or container cleanup cannot be proven safe, onboarding stops before the retry and prints manual cleanup guidance.
Image build, upload, TLS, provider, policy, dashboard, and inference failures stay on their existing error paths and do not trigger the GPU compatibility fallback.
Set NEMOCLAW_DOCKER_GPU_PATCH=1 to use only the compatibility path for diagnostics or older host compatibility.
Other legacy nonzero values keep that behavior through the v0.0.x release line and will be removed in v0.1.0.
Docker Desktop WSL compatibility route
Automatic GPU onboarding uses the compatibility path directly; it does not make a native attempt first.
The path creates the sandbox and then recreates the OpenShell-managed Docker container with NVIDIA GPU flags.
NEMOCLAW_DOCKER_GPU_PATCH=0 is ignored because this runtime requires the compatibility patch for GPU passthrough, and onboarding logs a warning when it is set.
To skip GPU passthrough entirely, rerun with --no-gpu or set NEMOCLAW_SANDBOX_GPU=0.
Jetson and Tegra compatibility default
Automatic GPU onboarding uses the compatibility path directly; it does not make a native attempt first.
The path recreates the OpenShell-managed Docker container with NVIDIA GPU flags and propagates the supplementary groups required for /dev/nvmap and /dev/nvhost-* access.
Use NEMOCLAW_DOCKER_GPU_PATCH=0 only for troubleshooting because it bypasses that group propagation and CUDA may not initialize.
Common compatibility-path recovery
If the compatibility attempt fails on any host, onboarding leaves the failed sandbox and diagnostic bundle in place so you can inspect the OpenShell and Docker state.
Starting with NemoClaw v0.0.43, the standard installer handles the /proc/<pid>/task/<tid>/comm permission case during this patch path.
If an older release fails direct GPU proof with that path and Permission denied, upgrade NemoClaw and rerun onboarding.
The output includes a cleanup command such as:
Fix the NVIDIA Container Toolkit or CDI configuration reported in the diagnostics, clean up the failed sandbox, then rerun onboarding.
If you do not need GPU access inside the sandbox, rerun with --no-sandbox-gpu.
If sandbox creation fails with CDI device injection failed: unresolvable CDI devices nvidia.com/gpu=all, the OpenShell gateway tried docker create --device nvidia.com/gpu=all and Docker could not resolve the CDI spec.
This injection happens inside the gateway, so NEMOCLAW_DOCKER_GPU_PATCH=0 does not bypass it.
Rerun with --no-gpu, or set NEMOCLAW_SANDBOX_GPU=0 and resume onboarding.
If onboarding reports OpenShell supervisor did not reconnect to the GPU-enabled container. even though the diagnostic bundle shows the patched container is running and healthy, the supervisor-reconnect wait is treating a transient Error phase (reported while the OpenShell host re-registers the new container) as fatal.
The reconnect wait debounces consecutive Error-phase polls before fast-failing, defaulting to fifteen consecutive polls of about 30 seconds in total.
Increase the debounce window with NEMOCLAW_DOCKER_GPU_SUPERVISOR_RECONNECT_ERROR_DEBOUNCE if your host needs more time to re-register the patched container, for example slow WSL2 + Docker Desktop setups.
Set it to a higher integer such as 30 (about 60 seconds) and rerun onboarding; the value is clamped to a minimum of 1.
If reconnect still fails after the GPU patch, NemoClaw attempts to restore the pre-patch CPU container before exiting.
When rollback succeeds, the output says the pre-patch sandbox was restored.
When rollback fails, the error says rollback failed and the pre-patch container was not restored, so inspect Docker state before retrying.
pip install fails with a system-packages error
Recent Ubuntu releases (including DGX Spark’s Ubuntu 24.04) mark the system Python install as externally managed, so pip install without a virtual environment fails.
Use a venv instead.
Avoid --break-system-packages unless you understand the risk, since it can break host tooling.
Port 3000 conflict with AI Workbench
NVIDIA AI Workbench’s Traefik proxy binds ports 3000 and 10000. If you run other services on Spark that expect port 3000, bind them to a different port.
Windows Subsystem for Linux
For environment setup steps, refer to Windows Prerequisites.
wsl --install --no-distribution returns Forbidden (403)
Check your network connectivity.
If you are behind a VPN, try reconnecting or switching to a different network.
If your network or Windows image blocks the online WSL installer, install WSL manually with Microsoft’s offline install guidance.
Download the latest official WSL .msi package from the Microsoft WSL releases page, choose the matching .x64.msi or .arm64.msi, install it, reboot if Windows requests it, then rerun wsl --status.
Bootstrap says “Windows Subsystem for Linux is not fully installed”
The bootstrap script checks wsl --status before it installs or opens Ubuntu.
If Windows reports that the WSL runtime is not installed, the script attempts wsl --install --no-distribution automatically.
If the repair command succeeds and WSL reports that the changes require a reboot, reboot and let the bootstrap resume after sign-in.
If the repair command succeeds but WSL still cannot be verified and the output does not request a reboot, follow the printed repair guidance instead of rebooting by default.
If the repair command fails, follow the printed repair steps.
If the repair command returns Forbidden (403) or remains blocked, install WSL manually with Microsoft’s offline install guidance.
Download the latest official WSL .msi package from the Microsoft WSL releases page, choose the matching .x64.msi or .arm64.msi, install it, reboot if Windows requests it, then rerun the bootstrap script.
Use the same repair flow if the bootstrap says “Windows Subsystem for Linux could not be verified” and reports a nonzero wsl --status exit code.
Bootstrap says “Windows reports that WSL 2 cannot start yet”
The bootstrap script attempts wsl --install --no-distribution automatically when wsl --status reports that WSL 2 cannot start.
If the repair command succeeds, reboot when prompted and let the bootstrap resume after sign-in.
If the message persists after repair and reboot, enable virtualization in firmware and confirm that the Virtual Machine Platform optional component is enabled.
Manual WSL installation only helps when the WSL runtime is missing or the online installer is blocked.
wsl -d Ubuntu says “There is no distribution with the supplied name”
The Ubuntu package was installed with --no-launch but never registered, or Windows finished the install command before the distribution appeared in wsl -l.
When this happens during the NemoClaw bootstrap, the script prints a sanitized WSL install output block.
PowerShell transcript headers, footers, temporary transcript paths, and status-file paths are redacted before display so you can paste the useful WSL output into a bug report with less local machine metadata.
If the sanitized output says a reboot is required, reboot and rerun the bootstrap.
If it does not request a reboot, register the distro manually or reinstall without --no-launch:
docker info fails inside WSL
Confirm that Docker Desktop is running and that WSL integration is enabled for Ubuntu (Settings > Resources > WSL integration). Then restart WSL:
Windows-host Ollama is installed but not shown during onboarding
When NemoClaw runs inside WSL, it checks both the Windows-host Ollama HTTP endpoint and the Windows ollama.exe process.
If Ollama is installed but the daemon is not reachable through host.docker.internal:11434, the wizard should still offer a start or restart action.
If the Windows-host option does not appear, confirm that PowerShell interop is enabled in WSL and that Windows can locate Ollama:
If the process is missing, start Ollama from Windows and rerun onboarding.
If the process exists but the endpoint is unreachable, use the restart action when the wizard offers it, or restart Ollama from Windows with OLLAMA_HOST=0.0.0.0:11434.
Ollama inference fails or hangs in WSL
Ollama configures context length based on your hardware.
On some GPUs (for example RTX 3500), the default context length is not sufficient for OpenClaw.
During onboarding, NemoClaw raises loaded-model context lengths below 16384 to 16384 when NEMOCLAW_CONTEXT_WINDOW is unset.
Set the variable manually when you need a different value or when you run Ollama outside the managed onboarding path.
Force a larger context length:
Verify that Ollama inference works:
Replace <model-id> with the model you selected during onboarding (for example qwen3.5:4b).
If ollama serve fails with Error: listen tcp 127.0.0.1:11434: bind: address already in use, check whether Ollama is configured for automatic startup:
If it is active, stop it first, then start with the custom context length:
For additional troubleshooting, refer to the Windows Setup page. For first-time OpenClaw setup, refer to the Quickstart.
Podman
Podman is not a tested runtime. OpenShell officially documents Docker-based runtimes only. If you encounter issues with Podman, switch to a tested runtime (Docker Engine, Docker Desktop, or Colima) and rerun onboarding.
Brev
For Brev setup instructions, refer to Brev Web UI.
Most OpenClaw skills show as blocked
After deploying NemoClaw on Brev, the Skills page in the OpenClaw gateway dashboard shows most bundled skills with a blocked status.
Only three skills are available by default: healthcheck, skill-creator, and weather.
Skills are blocked for one of three reasons.
- The skill requires a macOS-only binary (
memo,remindctl,grizzly, and similar) that is not available on the Linux (GCP) instance Brev provisions. - The skill requires a CLI binary that is not pre-installed in the sandbox image, such as
ghfor the GitHub skill. - The skill requires API credentials that have not been configured, such as a Notion API key or Discord bot token.
Skills that require macOS-only binaries cannot be enabled on Brev. Skills that require additional CLI binaries require a custom sandbox image rebuild.
For credentials, use the supported host-side setup flow.
Rerun onboarding for inference or web search credentials, or use nemoclaw <name> channels add <telegram|discord|slack|wechat|whatsapp> for messaging channels.
To add a binary to the sandbox image, update the sandbox Dockerfile.base to install the required package, then rebuild:
After the rebuild completes, return to the Skills page to confirm the skill status has changed from blocked to ready.
openclaw config set fails with a permission error on Brev
When the sandbox config has been locked from the host, openclaw.json is owned by root and mounted read-only inside the sandbox.
Running openclaw config set inside the sandbox then returns:
In the default sandbox state, openclaw.json is writable by the sandbox user.
If you see this error, use the host-side config command instead:
Refer to Commands for the full list of supported configuration keys.
OpenClaw dashboard is unreachable after extended uptime on Brev
After leaving NemoClaw running for an extended period on Brev, the OpenClaw dashboard may return ERR_CONNECTION_RESET or fail to load in the browser.
The agent may still respond on messaging channels such as Telegram or Slack while the dashboard is unreachable.
Back up your workspace first
Take a snapshot before running onboard to protect your workspace files.
Re-run onboarding to restore dashboard connectivity:
Depending on current sandbox state, onboarding may prompt before recreating resources.
Skill install buttons do not work on Brev
Clicking Install on a skill in the OpenClaw gateway dashboard on Brev shows no response or fails silently.
Skill installation runs against the sandbox environment. Installing packages on the Brev host does not make them available inside the sandbox. To install a skill dependency, add it to the sandbox image and rebuild:
After the rebuild completes, return to the Skills page to confirm the skill is ready.