> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemoclaw/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemoclaw/_mcp/server. # Architecture Details > Learn how NemoClaw combines a host CLI, sandbox integration layer, and versioned blueprint to run compatible agents in controlled OpenShell sandboxes. NemoClaw combines a host CLI, an in-sandbox integration layer, and a versioned YAML blueprint that defines the sandbox image, policies, and inference profiles applied through OpenShell. ## System Overview NVIDIA OpenShell is a general-purpose agent runtime. It provides sandbox containers, a credential-storing gateway, inference proxying, and policy enforcement, but it has no opinions about what runs inside. NemoClaw is an opinionated reference stack built on OpenShell that handles what goes in the sandbox, prepares agent-specific integration, and makes the setup accessible. ```mermaid graph LR classDef nemoclaw fill:#76b900,stroke:#5a8f00,color:#fff,stroke-width:2px,font-weight:bold classDef openshell fill:#1a1a1a,stroke:#1a1a1a,color:#fff,stroke-width:2px,font-weight:bold classDef sandbox fill:#444,stroke:#76b900,color:#fff,stroke-width:2px,font-weight:bold classDef agent fill:#f5f5f5,stroke:#e0e0e0,color:#1a1a1a,stroke-width:1px classDef external fill:#f5f5f5,stroke:#e0e0e0,color:#1a1a1a,stroke-width:1px classDef user fill:#fff,stroke:#76b900,color:#1a1a1a,stroke-width:2px,font-weight:bold USER(["👤 User"]):::user subgraph EXTERNAL["External Services"] INFERENCE["Inference Provider
NVIDIA Endpoints · OpenAI
Anthropic · Ollama · vLLM · Model Router"]:::external INTEGRATIONS["Integration APIs
MCP · GitHub · package indexes"]:::external INTERNET["Internet
PyPI · npm · GitHub · APIs"]:::external end subgraph HOST["Host Machine"] subgraph NEMOCLAW["NemoClaw"] direction TB NCLI["CLI + Onboarding
Guided setup · provider selection
credential validation · deploy"]:::nemoclaw BP["Blueprint
Hardened Dockerfile
Network policies · Presets
Security configuration"]:::nemoclaw MIGRATE["State Management
Migration snapshots
Credential stripping
Integrity verification"]:::nemoclaw end subgraph OPENSHELL["OpenShell"] direction TB GW["Gateway
Credential store
Inference proxy
Policy engine
Device auth"]:::openshell OSCLI["openshell CLI
provider · sandbox
gateway · policy"]:::openshell INTEG["Managed integrations
MCP · credentials · policy"]:::openshell subgraph SANDBOX["Sandbox Container 🔒"] direction TB AGENT["Compatible Agent
Selected managed runtime"]:::agent PLUG["NemoClaw Integration
Managed configuration
and runtime context"]:::sandbox end end end USER -->|"nemohermes onboard
nemohermes connect"| NCLI USER -->|"Tasks and approvals"| NCLI NCLI -->|"Orchestrates"| OSCLI BP -->|"Defines sandbox
shape + policies"| SANDBOX MIGRATE -->|"Safe state
transfer"| SANDBOX AGENT -->|"Inference requests
no credentials"| GW GW -->|"Proxied with
credential injected"| INFERENCE INTEGRATIONS -->|"External APIs"| INTEG INTEG -->|"Policy-gated access"| AGENT AGENT -.->|"Policy-gated"| INTERNET GW -.->|"Enforced by
gateway"| INTERNET ``` ## Deployment Topology The logical diagram above shows how components relate. This section shows what actually runs where on the host. NemoClaw's default Docker-driver topology does not place the sandbox in an embedded k3s cluster. Only the default gateway port `8080` uses a NemoClaw-managed service. On Linux, NemoClaw prefers the upstream package-managed `openshell-gateway.service`. Before adoption, NemoClaw asks systemd for the unit's effective `ExecStart`. When systemd returns the service identity, NemoClaw requires trusted package paths for both the unit and its effective binary. It then checks the effective binary's version against the blueprint range (`min_openshell_version` through `max_openshell_version`). This check prevents a supported user-local OpenShell install from hiding an out-of-range binary selected by the package unit's absolute `ExecStart`. NemoClaw rejects a known version below the minimum or a known stable version above the maximum. An above-maximum development build remains eligible when `NEMOCLAW_OPENSHELL_CHANNEL=dev`. An undetermined version or an unsupported version stops package-unit adoption. When the systemd identity query reports a recognized unavailable user manager, NemoClaw checks the standard user unit paths before standalone fallback. It checks `.wants`, `.requires`, and `.upholds` activation links for both gateway service names. An activation path blocks fallback because its service can later claim port `8080`. An active `SYSTEMD_UNIT_PATH` override also blocks fallback because NemoClaw cannot verify its activation state. Other identity-query failures stop package-unit adoption. NemoClaw stops when systemd identifies an untrusted unit or effective binary. For a version rejection, NemoClaw reports the version once and blocks another gateway lifecycle. Identity and path-trust failures also block another gateway lifecycle. After `systemctl --user daemon-reload` and systemd identity validation, startup checks the effective binary's version again before any service-state mutation. If a known version is now unsupported, startup does not stop, enable, or restart the package service. Onboarding stops instead of starting a standalone gateway. For tarball installs, the NemoClaw installer stages a marked user-level `nemoclaw-openshell-gateway.service` based on the upstream unit. Onboarding validates and reuses a healthy selected service. It enables or restarts the service when startup or verified runtime drift requires it, then checks gateway health. The marked service generates the local OpenShell mTLS bundle and reads `$XDG_CONFIG_HOME/openshell/gateway.env`, or `~/.config/openshell/gateway.env` when `XDG_CONFIG_HOME` is not absolute. NemoClaw preserves unrelated environment entries, accepts `DOCKER_HOST` only for an absolute local `unix://` socket, and refuses foreign or symlinked managed files. For ordinary startup or health failures, NemoClaw prints the `journalctl` log command and attempts the standalone fallback. Unknown inspection, trust, and version failures stop onboarding. On Apple Silicon macOS, Homebrew makes the official OpenShell formula authoritative. The installer stages the checksum-pinned formula and runs installation inside a formula-scoped Homebrew trust boundary. Onboarding uses the same boundary for formula inspection and each service start or stop operation. Each operation verifies the staged formula, removes any inherited formula trust, grants temporary trust only to `nvidia/openshell/openshell`, and removes that trust when the operation finishes. An inability to grant or remove trust is a hard failure. If the service fails inspection, startup, shutdown, or its health check, NemoClaw prints the formula log command and stops so Homebrew remains lifecycle authority. A host without Homebrew, or with no staged formula and no installed OpenShell keg, uses the standalone macOS gateway. A legacy installation, missing pin, changed formula, failed Homebrew operation, or invalid official-tap identity stops with repair guidance instead of changing lifecycle authority. Rerunning the standard NemoClaw installer downloads and stages the pinned formula before onboarding is retried: ```bash curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash ``` On both platforms, standalone startup requires exclusive ownership of the gateway port. A positively untrusted upstream package service is declined as described above. Trust failures in a marked NemoClaw service or Homebrew formula, and unsafe environment configuration, remain hard failures. NemoClaw-managed gateways on custom ports remain detached and separate from the default service. An externally supervised gateway can use any matching configured port; its declared supervisor retains lifecycle authority. In both Docker-driver modes, the sandbox is a Docker container, not a Kubernetes pod. Entrypoint supervisors create the in-container `/tmp/nemoclaw-gateway-local` marker only when they actually launch an in-container gateway, and they normally keep it present while that supervisor is active. On normal exits, handled `SIGTERM`/`SIGINT`, startup failures, and shell `errexit` termination through the `EXIT` trap, the supervisor removes the marker on a best-effort basis so the Docker health check does not keep trusting a stale gateway PID. Terminal runtimes may not write it. NemoClaw does not treat sandbox environment hints such as `OPENSHELL_DRIVERS` as authoritative for gateway ownership. Legacy non-Docker-driver installs still use the k3s-based gateway path. In that topology, the `openshell-cluster-nemoclaw` container runs an embedded k3s cluster that includes the OpenShell gateway, an `agent-sandbox-controller` workload, and a Kubernetes custom resource definition named `sandboxes.agents.x-k8s.io`. Each NemoClaw sandbox appears as a `Sandbox` custom resource in the `openshell` namespace, and the controller reconciles that resource into the corresponding agent pod. For example, `kubectl get sandboxes.agents.x-k8s.io -n openshell` inside the legacy cluster container lists the sandbox resources, and `kubectl describe pod -n openshell ` reports `Controlled By: Sandbox/`. That Kubernetes resource path is a legacy implementation detail of the non-Docker-driver gateway, and it is not present in the default Docker-driver topology. The diagram below shows the standard Docker-driver topology. ```mermaid graph TB classDef host fill:#fff,stroke:#76b900,stroke-width:2px,color:#1a1a1a,font-weight:bold classDef cli fill:#76b900,stroke:#5a8f00,color:#fff,stroke-width:2px,font-weight:bold classDef docker fill:#2496ed,stroke:#1577c2,color:#fff,stroke-width:2px,font-weight:bold classDef gateway fill:#1a1a1a,stroke:#1a1a1a,color:#fff,stroke-width:2px,font-weight:bold classDef sandbox fill:#444,stroke:#76b900,color:#fff,stroke-width:2px classDef external fill:#f5f5f5,stroke:#e0e0e0,color:#1a1a1a,stroke-width:1px subgraph HOST["Host machine · Linux / Apple Silicon macOS / DGX Spark / DGX Station"] direction TB CLI["nemohermes CLI
bin/nemoclaw.js → dist/
onboard · connect · status · logs"]:::cli GW["OpenShell gateway
host process by default
credential store · lifecycle · L7 proxy"]:::gateway subgraph DOCKER["Docker daemon"] direction TB SANDBOX["Sandbox container 🔒
Landlock + seccomp + netns
Compatible agent + NemoClaw integration"]:::sandbox end end INFER["Inference provider
NVIDIA Endpoints · OpenAI
Anthropic · Ollama · vLLM · Model Router"]:::external CLI -->|"openshell CLI
(orchestrates)"| GW GW -->|"creates/recreates
Docker-driver sandbox"| SANDBOX SANDBOX -->|"inference requests
placeholder credentials"| GW GW -->|"egress with real credentials
injected at the L7 proxy"| INFER class HOST host class DOCKER docker class GW gateway class SANDBOX sandbox ``` Layering from top to bottom: | Layer | Runs as | Role | | ------------------ | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | Host CLI | Host process (`nemohermes` on Node.js) | Orchestrates OpenShell via `openshell` CLI calls. | | OpenShell gateway | Host process by default; optional Linux compatibility container when the gateway binary needs a newer host ABI | Hosts the credential store, owns sandbox lifecycle coordination, and provides the L7 proxy. | | Docker daemon | Host service | Runs the Docker-driver sandbox container and, on affected Linux hosts, the optional gateway compatibility container. | | Sandbox container | Docker container | Runs the selected compatible agent and NemoClaw integration under Landlock + seccomp + netns. | | OpenShell L7 proxy | Gateway process | Intercepts agent egress and rewrites `Authorization` headers (Bearer/Bot) and URL-path segments to inject the real credential at the network boundary. | NemoClaw never gives the sandbox a raw provider key. At onboard time it registers credentials with OpenShell's provider/placeholder system, and the L7 proxy substitutes the real value into outbound requests at egress. The CLI helper `isInferenceRouteReady` (in `src/lib/onboard.ts`) is a host-side readiness check used by the resume flow to decide whether the active route already covers the chosen provider and model. It is not a runtime component. For the DGX Spark-specific variant of this topology (cgroup v2, aarch64, unified memory), refer to the [NVIDIA Spark playbook](https://build.nvidia.com/spark/nemoclaw). ## NemoClaw Agent Integration NemoClaw integrates with each supported agent through a runtime layer that adapts the agent to OpenShell-managed providers, policies, and sandbox state. The concrete files differ by agent because each runtime has its own plugin system, config format, state layout, and startup command. | Agent | Integration files | Runtime behavior | | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Hermes | `agents/hermes/manifest.yaml`, `agents/hermes/plugin/plugin.yaml`, `agents/hermes/generate-config.ts`, `agents/hermes/config/`, and `agents/hermes/start.sh` | Declares the Hermes agent contract, installs the NemoClaw Hermes plugin, writes `/sandbox/.hermes/config.yaml` and `/sandbox/.hermes/.env`, and launches `hermes gateway run` behind the OpenShell proxy. | The Hermes integration follows the generic agent-manifest path instead of the OpenClaw plugin package path. The manifest declares Hermes' binary, health probe, config directory, state directories, and OpenAI-compatible API endpoint. Messaging channel availability is declared by each channel manifest's `supportedAgents` list under `src/lib/messaging/channels/`, not by the Hermes agent manifest. The build-time config generator turns NemoClaw onboarding choices into Hermes YAML and environment files, and the Hermes plugin manifest exposes NemoClaw tools and an `on_session_start` hook. ## NemoClaw Blueprint The blueprint is a versioned YAML package with its own release stream. The runner resolves, verifies, and applies the blueprint through the OpenShell CLI. The blueprint defines the sandbox shape, default policies, and inference profiles; the runner performs the OpenShell operations. ```text nemoclaw-blueprint/ ├── blueprint.yaml Manifest: version, profiles, compatibility ├── model-specific-setup/ Agent-scoped model/provider compatibility manifests ├── router/ Model Router config and routing engine ├── policies/ │ └── presets/ Shared policy presets ``` The blueprint schema and runner enforce these name constraints before the runner invokes OpenShell: | Field | Constraint | | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `components.sandbox.name` | Use 1–19 lowercase letters, numbers, or single internal hyphens, starting with a letter and ending with a letter or number. Consecutive hyphens (`--`) are not allowed. | | `components.inference.profiles..provider_name` | Use 1–128 letters, numbers, dots, underscores, or hyphens, starting with a letter. | Hermes keeps its agent-owned image, plugin, config, entrypoint, and policy additions under `agents/hermes/`. The default Hermes policy starts from `agents/hermes/policy-additions.yaml`. The current blueprint runner implementation lives in the `nemoclaw/` TypeScript package: ```text nemoclaw/src/blueprint/ ├── runner.ts CLI runner: plan / apply / status / rollback ├── ssrf.ts SSRF endpoint validation (IP + DNS checks) ├── private-networks.ts Shared private-network block list loader for SSRF checks ├── snapshot.ts Migration snapshot / restore lifecycle ├── state.ts Persistent run state management ``` ### Blueprint Lifecycle ```mermaid flowchart LR A[resolve] --> B[verify digest] B --> C[plan] C --> D[apply] D --> E[status] ``` 1. Resolve. The integration layer locates the blueprint artifact and checks the version against the OpenShell and agent runtime constraints in `blueprint.yaml`. 2. Verify. The integration layer checks the artifact digest against the expected value. 3. Plan. The runner determines what OpenShell resources to create or update, such as the gateway, providers, sandbox, inference route, and policy. 4. Apply. The runner executes the plan by calling `openshell` CLI commands. 5. Status. The runner reports current state. ### Experimental Runtime Identity Runtime identity is an experimental capability of the direct OpenClaw blueprint runner. The maintained onboarding path for this agent does not consume the component. ## Sandbox Environment Normal NemoClaw onboarding builds from the [`ghcr.io/nvidia/nemoclaw/sandbox-base`](https://github.com/NVIDIA/NemoClaw/pkgs/container/nemoclaw%2Fsandbox-base) base image and layers the NemoClaw runtime Dockerfile on top. Inside the sandbox: * The selected compatible agent runs with the NemoClaw integration layer installed or generated for that agent. * Inference calls are routed through OpenShell to the configured provider. * Network egress is restricted by the baseline policy for the selected agent profile. * Filesystem access is confined to `/sandbox` and `/tmp` for read-write access, with system paths read-only. * NemoClaw writes generated Hermes configuration into the sandbox, then the Hermes runtime exposes its own gateway and health surface. * The image exposes health checks for the managed Hermes runtime. * The image includes common runtime compatibility helpers such as Homebrew and a `python` to `python3` symlink for tools that still invoke `python`. ## Inference Routing Inference requests from the agent never leave the sandbox directly. OpenShell intercepts them and routes them to the configured provider: ```text Compatible agent (sandbox) ──▶ OpenShell gateway ──▶ Provider endpoint ``` When you select the Model Router provider, the OpenShell gateway routes to a host-side router process instead of a single upstream model. The router selects from the configured pool, then calls the upstream NVIDIA endpoint with the credential held outside the sandbox. Some model and provider combinations need agent-specific compatibility setup. NemoClaw keeps those declarations under `nemoclaw-blueprint/model-specific-setup//` so fixes for each supported agent can be tested and reviewed independently. ### Managed Inference Catalog Managed inference profiles use definitions under `managed-inference/presets/` and `managed-inference/recipes/`. The catalog compiler validates those definitions and includes them in the packaged runtime catalog. For a managed-cluster recipe, the `nodeCount` field declares cluster cardinality. Registered managed-cluster TypeScript adapters validate qualified topology and materialize each ranked node. Contributors can define a compatible managed-cluster profile that uses an existing execution contract with preset and recipe YAML. A managed-cluster profile that needs a different execution contract requires a registered adapter. Refer to [Choose an Inference Provider](../inference/learn-and-choose/choose-inference-provider) for provider configuration details. ## Provider Credential Storage Provider credentials live in the OpenShell gateway store, not on the host filesystem. NemoClaw never writes them to host disk. The OpenShell L7 proxy injects values at egress. Refer to [Credential Storage](../security/credential-storage) for the inspection, rotation, and migration flow. ## Host-Side State and Config NemoClaw keeps non-secret operator-facing state on the host rather than inside the sandbox. | Path | Purpose | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `~/.nemoclaw/sandboxes.json` | Registered sandbox metadata for the default gateway port, including the default sandbox selection. | | `~/.nemoclaw/gateways//` | Segregated host state root (its own registry, snapshots, and legacy credential-migration files) for a non-default `NEMOCLAW_GATEWAY_PORT`. On upgrade, rows and related state move out of the legacy shared root only when their recorded gateway identity matches the selected port. Provider credentials remain in the OpenShell gateway store. The Ollama auth proxy token, backend URL, PID, and status files stay in the top-level `~/.nemoclaw/` location for every gateway port, because one proxy on the host serves them all. The default gateway port uses the top-level `~/.nemoclaw/` location, so existing single-gateway hosts are unchanged. | The following environment variables configure optional services and local access. | Variable | Purpose | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `NEMOCLAW_GATEWAY_PORT` | Optional host-side gateway port override for an independent OpenShell gateway and port-scoped NemoClaw state root. Supported for OpenClaw, Hermes, and Deep Agents. | | `TELEGRAM_BOT_TOKEN` | Telegram bot token you provide before `nemohermes onboard`. OpenShell stores it in a provider; the sandbox receives placeholders, not the raw secret. | | `TELEGRAM_ALLOWED_IDS` | Comma-separated Telegram user or chat IDs for allowlists when onboarding applies channel restrictions. | | `SLACK_BOT_TOKEN` | Slack bot token (`xoxb-...`) you provide before `nemohermes onboard`. Stored as an OpenShell provider; never passed directly to the sandbox. | | `SLACK_APP_TOKEN` | Slack app-level token (`xapp-...`) required for Socket Mode. Stored alongside `SLACK_BOT_TOKEN` during onboarding. | | `SLACK_ALLOWED_USERS` | Comma-separated Slack member IDs for DM and channel `@mention` user allowlisting. | | `SLACK_ALLOWED_CHANNELS` | Comma-separated Slack channel IDs where channel `@mention` events are enabled (e.g. `C012AB3CD,C987ZY6XW`). Baked into the sandbox image at build time. Combine with `SLACK_ALLOWED_USERS` to restrict both channel and member. | For normal setup and reconfiguration, prefer `nemohermes onboard` over editing these files by hand.