Set Up vLLM | NVIDIA NemoClaw

NemoClaw can connect to an existing vLLM server or, when Docker is available on PATH, install and start a managed vLLM container on supported NVIDIA GPU hosts. It also provides a Deferred N1x preview pending a physical NemoClaw Express E2E run.

NemoClaw forces the Chat Completions API path for vLLM. The vLLM /v1/responses endpoint does not run the configured tool-call parser, so tool calls can arrive as raw text.

Local vLLM does not require authentication by default. Existing-server and single-host managed-vLLM paths use NEMOCLAW_VLLM_PORT, which defaults to 8000, on host loopback for validation and on the OpenShell Docker bridge for sandbox traffic. After onboarding, status and doctor probe an authenticated managed single-host vLLM container on its published loopback port. For a bearerless server, they use the port in the sandbox’s recorded http://host.openshell.internal:<port>/v1 route. When that route is valid, no later port environment override is required. For vLLM, connect runs these host probes only when it repairs the route. A missing or invalid recorded route makes a bearerless probe fall back to NEMOCLAW_VLLM_PORT, which defaults to 8000. Use a host firewall with default-deny inbound rules. Allow the configured TCP port only from the OpenShell Docker subnet to its gateway address, keep loopback access, and deny the port on every other interface. Do not expose the port to your LAN or the internet.

An authenticated fixed profile publishes the configured host port on exactly two host addresses: 127.0.0.1 and the private IPv4 gateway of the openshell-docker bridge. The container continues to listen on its recipe-owned internal port 8000. It never publishes that host port on 0.0.0.0 or ::. The provider route remains https://inference.local inside the sandbox, and OpenShell reaches the server through the private bridge binding. The managed Muse Glimmer profile uses the same bearer-protected listener and two-address publication boundary.

The qualified two-DGX Spark and two-DGX Station paths use bearer-protected /v1 routes and Docker host networking on both runtime containers; network isolation remains required for other endpoints and distributed traffic. Follow Set Up vLLM on Two DGX Sparks or Set Up vLLM on Two DGX Stations for their isolation requirements.

Use an Existing Server

When vLLM is already running on localhost:${NEMOCLAW_VLLM_PORT:-8000}, NemoClaw detects it and queries /v1/models for the loaded model. On N1x, decline or disable Express, or set NEMOCLAW_PROVIDER=vllm, to supply explicit standard-onboarding intent.

Run onboarding.

nemoclaw onboard

Select Local vLLM. On generic hosts, the entry includes an experimental label. On N1x, the existing-server route remains unvalidated even after standard-onboarding intent admits it. NemoClaw validates the detected endpoint and records the model reported by /v1/models.

When you request a vLLM serving profile, the detected server must report the profile’s served model alias or a safe root that matches the profile’s declared model. NemoClaw stops onboarding when neither identifier matches, rather than recording a route that the profile does not declare. Stop the server and rerun the original command, or unset NEMOCLAW_SERVING_PRESET and NEMOCLAW_PROVIDER before you start fresh onboarding to keep the detected model.

On hosts other than N1x, when an install request without a managed GPU selection encounters an already-running server, NemoClaw selects that server instead of starting another container. On N1x, explicit managed-preview intent keeps the managed entry and stops onboarding when another vLLM server occupies the configured port. Stop the server, then rerun managed onboarding.

NEMOCLAW_PROVIDER=install-vllm \
  nemoclaw onboard --non-interactive

If the requested registry model and the served id differ, NemoClaw accepts the served id only when /v1/models reports a safe root that exactly matches the registry model. NemoClaw rejects mismatched or unverifiable roots, and it still requires the served identity for a newly started NemoClaw-managed container and for an existing shared-gateway route. When vLLM exposes runtime metadata such as max_model_len, NemoClaw uses that value for the configured context window unless you set NEMOCLAW_CONTEXT_WINDOW.

Start vLLM with the model you want before onboarding because NemoClaw uses the model already served by the endpoint.

On DGX Spark, an existing vLLM server is operator-managed. NemoClaw warns when vLLM metadata heuristically indicates a large checkpoint without reported quantization, does not identify the underlying model size, or reports an unusually long context window. These configurations can leave too little unified-memory headroom for the host and surface as NVRM NV_ERR_NO_MEMORY, SSH loss, or a hard host freeze under agent/tool load.

Prefer the managed Spark vLLM path when you want NemoClaw to use the validated Spark recipe, or restart your existing server with lower --gpu-memory-utilization, --max-model-len, --max-num-seqs, and --max-num-batched-tokens before onboarding. For full diagnostics and recovery steps, refer to DGX Spark troubleshooting.

Configure Structured Tool Calls

Start vLLM with automatic tool choice and the tool-call parser that matches the model family when the model serves a tool-using agent. For a Hermes 3 style model, use this command shape.

vllm serve /models/Hermes-3-Llama-3.1-8B \
  --served-model-name hermes-3-llama-3.1-8b \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --host 0.0.0.0 \
  --port 8000

Use the parser required by your selected model instead of copying hermes for another model family. Without both --enable-auto-tool-choice and a matching parser, tool calls can appear as raw assistant text instead of structured tool_calls. The --host value in this operator-run example controls the listener inside that server process. For the authenticated fixed profile, evaluate host exposure from its published Docker port bindings as described in Install a Fixed vLLM Profile.

Install or Start Managed vLLM

Managed install/start is Docker-backed and requires Docker on PATH. On a Docker-less native Podman host, attach an already-running local vLLM server or install Docker before selecting managed vLLM. NemoClaw displays the Install vLLM or Start vLLM entry by default on DGX Spark and DGX Station. N1x displays the same entry with a Deferred preview label. Generic Linux NVIDIA GPU hosts require NEMOCLAW_EXPERIMENTAL=1 or NEMOCLAW_PROVIDER=install-vllm before the managed entry appears.

Discover Serving Profiles

List the serving profiles installed with NemoClaw before you select a managed inference configuration.

nemoclaw profiles list

The command evaluates the catalog against the current host without downloading a model or changing host, gateway, inference, or sandbox resources. Each entry includes a stable ID, display name, backend, model, topology, selection mode, support state, estimated downloads, and an incompatibility reason when applicable. Add --json when a script needs the same fields as structured output.

Select a Serving Profile

Pass a compatible profile’s stable ID to onboarding.

nemoclaw onboard --profile <profile-id>

The same profile selector works with interactive and non-interactive onboarding. NemoClaw selects the inference provider required by the profile’s backend, so onboarding does not show the provider menu. If the profile’s backend has no corresponding inference provider, onboarding exits before it changes runtime resources. NemoClaw rejects unknown, ambiguous, disabled, incompatible, or conflicting selections before image or model downloads begin. Before confirmation, the review screen shows the resolved profile and recipe IDs, declared model, served model alias, immutable runtime image, support state, and estimated image and model downloads. Do not combine --profile with NEMOCLAW_PROVIDER, NEMOCLAW_MODEL, NEMOCLAW_VLLM_MODEL, NEMOCLAW_MANAGED_CLUSTER_PEERS, or NEMOCLAW_VLLM_EXTRA_ARGS_JSON overrides. If NEMOCLAW_SERVING_PRESET is already set, it must select the same stable profile ID; a different ID conflicts with --profile.

The profile selector is model-independent, so a catalog can add profiles without adding model-specific CLI flags. Omitting --profile preserves the existing provider and model defaults.

Select a Managed vLLM GPU

Use --vllm-gpu-device <index-or-uuid> to select the host GPU for a managed single-host vLLM container. The value must be a non-negative index or a full GPU-... UUID reported by nvidia-smi. This selection is independent of --sandbox-gpu-device, which controls direct GPU access inside the sandbox. NemoClaw applies the vLLM selection to Docker placement, compute-capability validation, and GPU-memory checks. It rejects this selector for an existing local vLLM provider or a managed multi-node topology. Only managed single-host vLLM installation accepts this selector.

nemoclaw onboard --profile <profile-id> --vllm-gpu-device <index-or-uuid>

Handle a Running vLLM Server

The single-host managed vLLM runtime is host-global. nemoclaw <name> destroy keeps the container while any registered sandbox in any gateway state root reports provider vllm-local. Local NIM uses the same provider identifier and counts as a possible consumer. After the last such sandbox is destroyed, an eligible destroy removes the verified NemoClaw-managed container and frees its GPU memory. When ownership or Docker state cannot be verified, NemoClaw retains recovery state and reports the reason instead of claiming completed retirement. Resolve the cause, then rerun the same destroy command. Pass --keep-vllm or set NEMOCLAW_KEEP_VLLM=1 to preserve the container for a later sandbox. Refer to CLI Commands for the cleanup conditions. Later standard onboarding normally reuses a server that is already listening on the configured vLLM port instead of replacing it automatically. When a managed GPU selection accompanies explicit managed-install intent, NemoClaw preserves that intent and reports the running-server conflict.

Choose one action:

To reuse the running server, omit --vllm-gpu-device and select Local vLLM. On N1x, decline or disable Express, or set NEMOCLAW_PROVIDER=vllm, before onboarding.
To change the GPU or port, inspect the sandbox registry for every gateway-port environment on the host. Run NEMOCLAW_GATEWAY_PORT=<port> nemoclaw list for port 8080 and for every non-default port represented by a numeric directory under ~/.nemoclaw/gateways/. Treat every sandbox that reports provider vllm-local as a possible consumer because the list output does not distinguish local vLLM from Local NIM. Continue only when no environment reports that provider; otherwise, keep the server running or move each possible consumer to another inference provider. Stop an operator-managed server through the system service, container, or process workflow that started it. For a NemoClaw-managed container, inspect its ID and labels before stopping it:
```
docker container inspect \
  --format '{{.Id}} {{json .Config.Labels}}' \
  nemoclaw-vllm
```
Continue only when the output includes "com.nvidia.nemoclaw.managed-vllm":"true" and does not include com.nvidia.nemoclaw.vllm-role, com.nvidia.nemoclaw.vllm-endpoint, or com.nvidia.nemoclaw.vllm-cluster. Do not stop the container when any distributed label is present; follow the matching multi-node vLLM guide instead. Stop that exact container ID:
```
docker container stop <container-id>
```
Set NEMOCLAW_VLLM_PORT before onboarding if you are changing the port. Rerun managed onboarding:
```
nemoclaw onboard --profile <profile-id> --vllm-gpu-device <index-or-uuid>
```

Accept the result when onboarding completes and nemoclaw list reports the new sandbox with provider vllm-local and the expected model. Onboarding does not stop or recreate a running vLLM server automatically because another sandbox can use it.

NemoClaw requires a resumed session to reuse the recorded GPU selector. A legacy in-progress session cannot add one, and a resumed session cannot change one. Run nemoclaw onboard --fresh when you need a different GPU selection.

If onboarding is interrupted, nemoclaw onboard --resume reuses the recorded catalog, preset, and recipe digests. It exits before runtime effects if the installed catalog changed instead of silently switching the recipe; use nemoclaw onboard --fresh --profile <profile-id> to review and start with the current definition. Legacy sessions that predate profile provenance still resume with their original behavior, but you cannot add --profile to one of those in-progress sessions.

In an interactive run, the managed path lists the supported model registry for your host profile. Press Enter to use the profile default or select another validated model with its registered vllm serve arguments.

The single-host path pulls the vLLM image, downloads model weights into ~/.cache/huggingface, and starts the nemoclaw-vllm container with the configured host port (8000 by default). For a bearer-protected profile, Docker publishes that port only on 127.0.0.1 and the private IPv4 gateway of the openshell-docker bridge. For a bearerless profile, Docker publishes the host port on all host interfaces and vLLM does not require NemoClaw’s managed bearer key. Before starting a bearerless profile, configure the host firewall to allow the port only from the OpenShell Docker subnet and loopback, and deny it on management, LAN, and internet-facing interfaces. The distributed DGX pages document their separate container names, endpoints, and lifecycle checks. The container carries NemoClaw ownership and immutable catalog labels so later onboarding, recovery, and uninstall runs can distinguish it from a bring-your-own server without trusting the container name alone. NemoClaw also records an owner-only, credential-free runtime receipt that binds those labels to the container ID and API-key fingerprint.

If an existing nemoclaw-vllm container does not carry that label, NemoClaw preserves it and asks you to remove or rename it before installing managed vLLM. On hosts other than N1x, if another vLLM server already occupies the configured vLLM port and no managed GPU selection accompanies explicit managed-install intent, NemoClaw selects the existing-server path and applies the model-identity checks described above. On N1x, NemoClaw stops managed onboarding and requires you to stop that server before trying the Deferred preview again. Except for the interrupted managed install described below, an occupied single-host port stops installation before NemoClaw persists a bearer key or records the selected model. For an interrupted install, it can remove and replace only a running nemoclaw-vllm container that has the NemoClaw ownership label and publishes container port 8000 to the requested host port on 127.0.0.1 or 0.0.0.0 through the selected local Docker connection. It refuses an unlabeled, stopped, distributed, ambiguous, differently mapped, or differently addressed container. For every other port holder, NemoClaw also stops before storage prompts, cache creation, image pull, or container start. Stop the process, then run onboarding again. Docker reports a bind failure if another process claims the port after this preflight. Managed profiles and model-specific recipes use immutable image digests:

DGX Spark and DGX Station models without a model-specific runtime use the linux/arm64 digest sha256:9204569b17ee4c0eff75194b8e6e458479c8aee18953b5ab9cf359fcdac659e2 with a compressed layer size of 9.60 GB under nvcr.io/nvidia/vllm:26.05.post1-py3. N1x uses the same image.
The DGX Spark Muse Glimmer recipe uses the linux/arm64 post-fix vLLM nightly digest sha256:b0e84e5f2b00a7268e4fdda332790ebd4bfb166b64757e166914753afaeee965 with a compressed layer size of 9.71 GB under vllm/vllm-openai. The image was built from vLLM commit 5a4c8d99242e9e069b604d0e9b969e77f7dd501d, which contains merged Muse Glimmer support commit 6adad08767583f52eb4d2122111af0bf638ed5e6 and the vLLM dependency fix from commit 90984ddbed27a09409506d6d6c0eea87f54b04b5. The image includes huggingface_hub 1.28.0, which preserves the pinned model revision when vLLM transfers configuration to the engine process. It pins Hugging Face revision d35cb79050f419c457611b1cee5c5d15b176f285 for the approximately 25.45 GB model download.
The DGX Spark Nemotron 3.5 Lightning recipe uses the linux/arm64 digest sha256:3af90144a0926e5c5fe46ee16e5201e763dd854538b9d7ce433755f11dadaf78 with a compressed layer size of approximately 12.69 GB under vllm/vllm-openai. It pins Hugging Face revision 0dcd680e5585c791728c83342b311d0a0026dbeb for the approximately 21.56 GB model download.
The DGX Station Nemotron 3 Ultra express recipe uses the multi-platform index digest sha256:0fec7ec5f3e6bc168e54899935fb0557da908a4832a1dbc88e2debcf2f889416 under vllm/vllm-openai:v0.22.0; on DGX Station, that index selects a linux/arm64 manifest with a compressed layer size of 10.67 GB. It also pins Hugging Face revision 183968f87ae4cedce3039313cac1fd43d112c578 for the approximately 352.38 GB model download.
Generic Linux arm64 hosts use sha256:447995cbb57e6c7cf792cab95e9852e5f62b5fb6d2f39e030fa4eda9a54eadb4 with a compressed layer size of 9.28 GB under nvcr.io/nvidia/vllm:26.03.post1-py3.
Generic Linux amd64 hosts use sha256:7be6c2f676c36059a494fe17254e69ae5c677535ba6191044e5fc8e42a91c773 with a compressed layer size of 8.93 GB under nvcr.io/nvidia/vllm:26.03.post1-py3.

Image pulls from nvcr.io require NGC registry login. The Station Nemotron 3 Ultra recipe pulls its vLLM image from Docker Hub instead. The two-Station page records the distributed recipe’s image and runtime versions.

The first run duration depends on image and model size, storage performance, and network throughput. Later runs reuse the cached image and model weights. The single-host managed container uses Docker’s --restart unless-stopped policy, so Docker restarts it unless an operator explicitly stops it.

Authenticate Hugging Face Downloads

Authentication is optional for public models, but a read token reduces exposure to anonymous HTTP 429 rate limiting during large downloads. Create a read token at Hugging Face Access Tokens, then export it in your shell before you start or restart onboarding. Do not paste the token into NemoClaw prompts or chat.

export HF_TOKEN="<your-read-token>"
nemoclaw onboard

NemoClaw reports whether the download is authenticated without displaying the token value. It passes the token only to the temporary model downloader (hf download) and does not retain it in onboarding state or the long-running vLLM container. NemoClaw also accepts HUGGING_FACE_HUB_TOKEN as a compatibility alias, but new setup should use HF_TOKEN. When neither variable is present, public-model downloads continue anonymously.

If hf download reports HTTP 429, export HF_TOKEN and resume onboarding. After an image-pull or managed-vLLM setup failure, resume retains the validated selected model and recorded GPU selector. A conflicting model, a changed GPU selector, or a selector introduced only during resume stops instead of changing the recorded install intent. The downloader reuses files already stored in ~/.cache/huggingface instead of starting the model download from an empty cache.

export HF_TOKEN="<your-read-token>"
nemoclaw onboard --resume

Authentication reduces anonymous rate-limit exposure but does not guarantee that a download cannot receive HTTP 429. Gated models still require license acceptance and a token before NemoClaw starts image or model downloads.

Check GPU Compute Capability

A quantized checkpoint does not load on a GPU whose compute capability is lower than the capability the checkpoint was built for. Before the image pull and the model download, NemoClaw reads the compute capability of each GPU that nvidia-smi reports. It compares the lowest reported capability against the selected model’s minimum.

On a mismatch, the managed install stops before those downloads and the error names both values.

  vLLM install failed: NVIDIA Nemotron-3 Nano 4B FP8 requires GPU compute capability 8.9 or newer, but this host reports 8.0. Serve this model on a newer GPU, or select a compatible model with NEMOCLAW_VLLM_MODEL.

Interactive onboarding returns to the inference-provider list so you can select another provider or model only when NEMOCLAW_PROVIDER does not pin a provider. When NEMOCLAW_PROVIDER=install-vllm pins managed vLLM, onboarding stops after the first failed attempt even if model selection was interactive. Correct the reported problem before rerunning or resuming, or unset NEMOCLAW_PROVIDER and rerun to choose another provider. Non-interactive onboarding exits.

Select a Managed Model lists the minimum for each registry entry. A registry entry with no declared minimum, such as an unquantized checkpoint, skips this check. When nvidia-smi reports no compute capability, NemoClaw continues and leaves the decision to vLLM.

Check Storage Before Downloads

For a cold managed-vLLM install, NemoClaw estimates local storage before the watched image pull starts. The estimate adds the pinned image’s compressed layer size, the digest catalog’s pre-measured unpacked layer size when available, the selected model’s Hugging Face file size, a 3 GiB model-download staging reserve, and a writable allowance.

For the DGX Station default, that is approximately 393.68 GB: 9.603 GB compressed image layers, 27.659 GB unpacked image layers, 352.381 GB model files, 3.221 GB model-download staging, and 0.816 GB writable allowance.

For Docker’s classic image store, NemoClaw checks the absolute DockerRootDir reported by docker info. For Docker’s containerd image store, it checks the top-level root from /etc/containerd/config.toml, or /var/lib/containerd when that configuration file is absent, and also checks DockerRootDir for pull staging. For model files, NemoClaw checks the filesystem backing ~/.cache/huggingface.

The check compares Docker image storage against the image bytes and compares model-cache storage against the model files, download staging, and writable allowance. When those paths share a filesystem, NemoClaw compares the shared capacity against the aggregate requirement. When NemoClaw cannot determine whether successful checks share a filesystem, it conservatively compares their aggregate requirement against the lowest reported capacity. NemoClaw does not move the Hugging Face cache to /raid or another fallback mount during this preflight.

NemoClaw recognizes an image cache hit only when Docker contains the platform digest. A cache hit skips the image portion of the estimate, but still checks the model-cache estimate before hf download. The explicit watched docker pull still runs against that immutable digest.

After an image pull, NemoClaw checks the model-cache estimate again before hf download. The model-download helper and the long-lived vLLM container use --pull=never, so an implicit pull cannot select another image.

NemoClaw treats /var/run/docker.sock and /run/docker.sock as the local default Docker socket paths. It cannot inspect daemon storage through a remote endpoint, a named non-default Docker context, or another Unix socket. When both DOCKER_CONTEXT and DOCKER_HOST are set, NemoClaw follows Docker CLI behavior and uses DOCKER_HOST. Before a registered model with pinned size metadata is downloaded, NemoClaw also checks the filesystem that contains the host Hugging Face cache.

When a Docker image-storage check is inconclusive, NemoClaw prints the reason, approximate requirement, and diagnostic commands, then continues without claiming that capacity is sufficient. An inconclusive model-cache check remains guarded because a registered model download can be hundreds of gigabytes. For an inconclusive model-cache check, interactive setup requires explicit confirmation, and non-interactive setup stops with guidance to rerun interactively.

When verified capacity is insufficient, the warning identifies the image, model, available capacity, approximate requirement, checked path and source, remediation, and diagnostic commands. Interactive setup continues only when you enter y or yes explicitly. Pressing Enter, entering n or no, or entering any other response stops before the guarded image or model download starts.

--yes and NEMOCLAW_YES=1 do not change managed vLLM storage-warning handling. Express installation and other non-interactive setup stop after a verified warning that Docker image storage or model-cache storage is insufficient, before the guarded download starts. Interactive setup still requires you to enter y or yes for a verified insufficient-capacity warning.

An inconclusive model-cache warning also requires explicit confirmation interactively, and non-interactive setup stops with guidance to rerun interactively. An inconclusive Docker image-storage check retains its existing automatic-continue behavior.

The model-cache guard applies when the selected registered recipe declares a pinned snapshot size. NemoClaw creates ~/.cache/huggingface as the host user and runs the one-shot download container with the host UID and GID, avoiding root-owned cache paths. The long-lived vLLM container mounts only the Hugging Face hub cache read-only and uses offline mode, so it cannot change host-cache ownership. Hugging Face credentials are forwarded only to that temporary downloader, not retained by the long-lived vLLM container.

A legacy root-owned or otherwise unwritable required model-cache path stops before the image pull and prints a scoped ownership-repair command; NemoClaw never changes its ownership automatically.

If image pull output stops making progress, a watchdog stops the stalled pull without imposing a fixed wall-clock limit on slow but active downloads. If vLLM does not become ready, NemoClaw prints a short tail of the container logs before exiting. Because the single-host managed container carries the --restart unless-stopped policy described above, a container that cannot serve its model restarts instead of exiting. While NemoClaw waits for the API, it also reads that container’s restart count. After the container restarts three times, NemoClaw stops the container and the install instead of waiting out the full load timeout.

When you start managed vLLM outside the installer express flow, NemoClaw uses these profile defaults:

Host profile	Default model
DGX Spark	`nvidia/Qwen3.6-35B-A3B-NVFP4`
N1x	`nvidia/Qwen3.6-35B-A3B-NVFP4`
DGX Station	`deepseek-ai/DeepSeek-V4-Flash`
Linux with an NVIDIA GPU	`nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`

For the managed one-host DGX Spark and N1x nvidia/Qwen3.6-35B-A3B-NVFP4 profiles, NemoClaw enables async scheduling and does not enable multi-token prediction (MTP) speculative decoding. DGX Spark automatically selects its Qwen recipe from the detected unified-memory capacity:

Detected capacity	Context tokens	Concurrent sequences	Batched tokens	GPU-memory utilization
At least 60 GB and less than 64 GB	32,768	1	4,096	0.5
At least 64 GB	262,144	4	8,192	0.4

The smaller recipe admits a nominal 64 GB DGX Spark that reports about 61.6 GB of usable unified memory. It loads weights with vLLM’s lazy safetensors strategy instead of fastsafetensors. It remains Experimental with software validation; physical 64 GB Spark qualification is pending. Unknown capacity or less than 60 GB does not qualify for either Qwen recipe. Before installation, NemoClaw displays the selected hardware profile and context limit. The Deferred N1x profile uses --gpu-memory-utilization 0.6, a 32768-token context window, 2 concurrent sequences, and a 4096-token batch limit. The two-sequence limit permits vLLM to schedule a compaction summary while an agent response uses the other sequence. Both profiles retain chunked prefill, prefix caching, and their registered parsers and acceleration backends.

NVIDIA Nemotron 3.5 Lightning is an additional explicit-only single-DGX Spark profile and does not change the Qwen default. Select its stable catalog ID after confirming that nemoclaw profiles list reports it as compatible.

nemoclaw onboard --profile vllm.dgx-spark-gb10.single.nemotron-3.5-lightning-30b-a3b-nvfp4

The profile serves the public nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 checkpoint through a pinned ARM64 vLLM image. Its profile limits for one concurrent sequence are a 65,536-token context window, a 4,096-token batch limit, 0.65 GPU-memory utilization, FP8 KV cache, and one-token MTP speculative decoding. It uses vLLM’s Python frontend because the pinned runtime’s Rust frontend does not register the required step3p5 reasoning and tool-call parsers. Physical single-DGX Spark validation of public revision 0dcd680e5585c791728c83342b311d0a0026dbeb covered direct chat, automatic and named structured tool calls, a ten-request release workload with input lengths from 32,768 through 41,984 tokens and 2,048 output tokens per request, OpenClaw chat and file tools, and a cached restart. Requests 2 through 10 averaged about 102 output tokens/s. At 0.65 utilization, the workload completed with about 33 GiB of host memory available and no NVIDIA kernel errors.

Experimental Nemotron 3.5 Lightning vLLM Profile

This profile is an explicit opt-in for one DGX Spark and remains Experimental. Promotion requires broader validation of the pinned Python frontend and the step3p5 reasoning and tool-call parsers. The fixed recipe intentionally rejects model and serve-argument overrides; use another catalog profile instead of modifying its validated resource boundary.

Reading Reasoning and Sizing max_tokens for Direct Callers

Nemotron 3.5 Lightning is a reasoning model that emits a hidden reasoning trace before its answer. Two behaviors surprise direct callers of the managed vLLM endpoint; the OpenClaw agent path is unaffected because it already requests a large budget.

Read reasoning from reasoning, not reasoning_content. The pinned vLLM runtime renamed the deprecated reasoning_content response field to reasoning, so choices[].message.reasoning_content is null even for prompts that reasoned heavily. The reasoning is not discarded. Read non-streaming reasoning from choices[].message.reasoning. For streaming responses, read choices[].delta.reasoning.
Send a generous max_tokens (at least 1024, and more for hard prompts). The reasoning trace counts against max_tokens. With a modest budget the model can spend the entire budget on reasoning before it reaches the answer, returning an empty content with finish_reason=length. A larger budget lets the answer follow the reasoning.

Muse Glimmer is an additional single-DGX Spark choice and does not change the Qwen default. Select its stable catalog ID after confirming that nemoclaw profiles list reports it as compatible.

nemoclaw onboard --profile vllm.dgx-spark-gb10.single.muse-glimmer-30b-nvfp4-w4a4

The profile serves Inferact/Muse-Glimmer-30B-NVFP4-W4A4 with the muse-glimmer alias. The registered recipe configures a 32,768-token context window, 1 concurrent sequence, a 4,096-token batch limit, and 0.75 GPU-memory utilization. It uses muse_glimmer for reasoning and tool-call parsing and sets --generation-config auto. The pinned post-merge runtime image was verified on one physical DGX Spark. The managed /v1/models endpoint reported muse-glimmer. A direct chat request returned 391 and no other text, and a reasoning request returned separated reasoning in message.reasoning. A required structured tool request returned a named tool call. A real OpenClaw TUI turn used one prompt and returned 391 through provider inference and model muse-glimmer. A managed stop and start preserved the image digest and managed inference route. The fixed profile does not enable DFlash, pass --quantization or --trust-remote-code, or install fastsafetensors. It rejects model and serve-argument overrides so its validated resource boundary remains reproducible.

Experimental Muse Glimmer vLLM Profile

This profile pins a post-merge vLLM nightly image and remains Experimental. Physical qualification covered only the bounded verification above. Vision and DFlash speculative decoding are disabled and outside this profile’s scope.

Use Explicit Linux x86_64 Profiles

Experimental Linux x86_64 profiles provide the Muse Glimmer and NVIDIA Nemotron 3.5 Lightning models on one NVIDIA GPU. They are explicit-only, so they do not change the generic Linux default or appear in the interactive model picker.

Before you select either profile, confirm that the host runs Linux x86_64 with Docker, the NVIDIA Container Toolkit, healthy CDI support, NVIDIA driver 580.65.06 or later, and at least 96 GB of GPU memory. Muse Glimmer also requires GPU compute capability 12.0 or later. NVIDIA Nemotron 3.5 Lightning requires GPU compute capability 9.0 or later.

List the compatible profiles, then pass one stable profile ID to onboarding:

nemoclaw profiles list
nemoclaw onboard --profile vllm.linux-amd64-nvidia.single.muse-glimmer-30b-nvfp4-w4a4

Use vllm.linux-amd64-nvidia.single.nemotron-3.5-lightning-30b-a3b-nvfp4 for NVIDIA Nemotron 3.5 Lightning. NemoClaw stops before image or model downloads when the profile does not match the host. The selected recipe uses a pinned image, fixed serving arguments, bearer authentication, and the managed host-local lifecycle. Do not combine --profile with NEMOCLAW_PROVIDER, NEMOCLAW_MODEL, NEMOCLAW_VLLM_MODEL, or NEMOCLAW_VLLM_EXTRA_ARGS_JSON.

Experimental Linux x86_64 Profiles

These profiles have bounded hardware validation and remain Experimental. They do not establish general support for other Linux x86_64 GPUs, models, images, or serving arguments.

NemoClaw creates or reuses the Muse Glimmer bearer key in the owner-only ~/.nemoclaw/dual-station-vllm-api-key host file before confirmation. The managed vLLM container reads the key while it runs, and the sandbox receives only the managed inference route. The key remains for a retry after cancellation, interruption, or a later preflight failure. A full uninstall removes it only after managed-runtime cleanup succeeds and no sibling gateway remains; gateway-scoped cleanup can preserve the host-global runtime and key for another gateway.

Use N1x Express

On an N1x host that matches the required identity, the Deferred Express preview selects one-host managed vLLM with nvidia/Qwen3.6-35B-A3B-NVFP4. The installer uses the existing managed-vLLM image, download, container, receipt, recovery, and uninstall lifecycle. The installer applies the existing CUDA and Container Device Interface (CDI) readiness checks before it downloads the image or model. The N1x memory, context, and batch limits above have reached the ready vLLM endpoint on one physical host. Physical validation of compaction with the two-sequence limit and the full Express onboarding E2E remain pending.

Upgrade an Existing N1x Managed vLLM Runtime

An existing nemoclaw-vllm container retains its original command after a NemoClaw update or ordinary sandbox rebuild. Replace an affected N1x managed container when its command contains --max-num-seqs 1.

Before replacement, follow Handle a Running vLLM Server to inspect every gateway environment and confirm that no other sandbox or distributed deployment uses the server. Stop only the exact verified managed container ID. If another sandbox can use the server, schedule coordinated downtime instead of continuing.

Rebuild the affected sandbox with explicit managed-vLLM intent.

NEMOCLAW_PROVIDER=install-vllm \
  nemoclaw my-assistant rebuild --yes

NemoClaw verifies the stopped container ownership, replaces that container with the current N1x recipe, and then continues the transactional sandbox rebuild. If managed-vLLM startup fails, the existing sandbox remains registered, but its stopped host inference server can remain unavailable. Correct the reported startup problem, then repeat the same rebuild command.

Verify that the replacement container uses the two-sequence limit.

docker container inspect \
  --format '{{json .Config.Cmd}}' \
  nemoclaw-vllm

Accept the result when the command contains --max-num-seqs 2.

NemoClaw requires all of this N1x identity evidence:

The host runs Linux on arm64.
/etc/fastos-release is a regular file of 1 through 4,096 bytes that is owned by UID 0 and GID 0.
The marker is not a symbolic link, grants no group or world write access, and contains exactly one NAME="N1x FASTOS" line.
A display-class PCI device has NVIDIA vendor ID 10de.

NemoClaw parses /etc/fastos-release as inert text and does not source it. The classifier does not pin a FastOS version, kernel version, driver version, firmware version, or GPU UUID. Generic DMI values, such as SKU 1 and JMJWOA-Generic, do not establish N1x identity by themselves.

N1x Express does not activate DGX Spark cluster discovery, the fixed catalog path, managed llama.cpp, or NVIDIA NIM. Accepting the preview prompt sets explicit managed-vLLM intent for the narrow pending-validation readiness exception. Declining the prompt continues with ordinary interactive onboarding. Setting NEMOCLAW_NO_EXPRESS=1 also enters ordinary onboarding when no provider is preselected or the explicit provider is recognized and is not Local NVIDIA NIM. An invalid or Local NVIDIA NIM provider remains blocked. Set a recognized provider other than Local NVIDIA NIM or install-vllm to enter ordinary onboarding directly. Set NEMOCLAW_PROVIDER=install-vllm when you need to bypass the prompt with the same explicit preview intent.

On a host that still meets the N1x identity checks, rebuild reuses the recorded route as explicit preview intent only when all of these conditions hold:

The sandbox registry records provider vllm-local.
The registry records model nvidia/Qwen3.6-35B-A3B-NVFP4.
If the registry records endpointUrl as null, NemoClaw derives http://host.openshell.internal:${NEMOCLAW_VLLM_PORT:-8000}/v1; otherwise, endpointUrl must equal that URL.
The registry records openshellDriver as docker.
The registry records endpointSource as onboard, or records deferredN1xManagedVllmAccepted: true. The latter accepts either a null endpoint or the canonical local endpoint.
A legacy record with both endpointUrl and endpointSource set to null can instead use the one-time NEMOCLAW_PROVIDER=install-vllm recovery decision below.
The replacement keeps the recorded provider and model, then re-derives the canonical local endpoint.
hostLocalInferenceReceipt is absent or null, or contains a canonical vLLM receipt for host.openshell.internal:${NEMOCLAW_VLLM_PORT:-8000} and the same model.

NemoClaw v0.0.109 can omit hostLocalInferenceReceipt or record it as null. NemoClaw v0.0.119 can record both endpointUrl and endpointSource as null without durable preview acceptance. For one rebuild of such a sandbox, repeat the explicit preview choice on the command that triggers the rebuild.

NEMOCLAW_PROVIDER=install-vllm \
  nemoclaw my-assistant channels add slack

A successful rebuild records deferredN1xManagedVllmAccepted: true, so later rebuilds do not require the environment variable. Matching sandboxes from either release remain eligible only when every other condition above passes. Except for the one-time v0.0.119 recovery above, you do not need to set NEMOCLAW_PROVIDER again for a matching rebuild. Any unmet condition stops readiness preflight before NemoClaw deletes the existing sandbox. A malformed, non-vLLM, or conflicting receipt also stops readiness preflight before deletion. This reuse waives only the pending physical-validation finding and does not change the Deferred status.

N1x Express Qualification Pending

Physical CUDA and CDI checks passed on one N1x host. The full NemoClaw Express E2E test is still pending, so the N1x platform status remains Deferred. Do not claim this path as supported until Platform Support reports a promoted status.

On DGX Station, accepting the installer express prompt selects NEMOCLAW_VLLM_MODEL=nemotron-3-ultra-550b-a55b. Express uses the single-Station Ultra recipe unless a trusted reciprocal pair qualifies for distributed serving. Refer to Set Up vLLM on Two DGX Stations for pair qualification, lifecycle receipts, runtime topology, and network controls.

On DGX Spark, Express keeps automatic managed-vLLM serving-profile selection as option 1, the default. It offers the fixed local vLLM profile as option 2. When you select option 2, the serving catalog supplies the pinned model, runtime image, container port, and serve arguments. The installer rejects provider and model overrides for that fixed profile. NEMOCLAW_VLLM_MODEL can select a catalog model only when the catalog resolves it to the matching fixed recipe, but the dedicated onboarder rejects a model that does not resolve to that recipe and all serve-argument overrides before it starts the managed container. Set NEMOCLAW_VLLM_PORT to select another host listener port. Explicit NEMOCLAW_MODEL or NEMOCLAW_VLLM_MODEL intent retains the customizable managed-vLLM path and skips the fixed-profile choice. If onboarding stops after you select option 2, nemoclaw onboard --resume reuses the recorded fixed profile instead of returning to automatic serving-profile selection. Refer to Choose a Local Inference Server for fixed-profile installation and verification. Refer to Set Up vLLM on Two DGX Sparks for automatic and explicit distributed-profile selection, profile requirements, and cleanup behavior.

To select the existing deepseek-v4-flash recipe while retaining the same one-confirmation express flow, run:

curl -fsSL https://www.nvidia.com/nemoclaw.sh | \
  NEMOCLAW_AGENT=openclaw \
  bash -s -- --station-deepseek

The flag requires an interactive terminal; in a curl | bash pipeline, /dev/tty must be available. Without terminal access, the installer stops before it installs Docker or build dependencies instead of silently continuing with another configuration.

The registered single-Station Ultra recipe tracks the official DGX Station deployment guide and configures the pinned model revision, CPU offload, 16 GB of shared memory, memory/stack ulimits, MTP speculative decoding, and the Nemotron reasoning and tool-call parsers.

Before managed vLLM setup on DGX Station, follow Prepare DGX Station to Install NemoClaw. For the current support status and direct GPU policy boundaries, see Platform Support.

Run Non-Interactive Onboarding

On DGX Spark, non-interactive onboarding can select local vLLM when NEMOCLAW_PROVIDER is unset and no provider is recorded for the sandbox. It reuses a running local vLLM server first. If no server is running, it selects the managed install or start entry. If neither local entry is available, it selects NVIDIA Endpoints. On DGX Station and other hosts, an unset provider keeps NVIDIA Endpoints as the automatic default.

Use an already-running server.

NEMOCLAW_PROVIDER=vllm \
  nemoclaw onboard --non-interactive

On N1x, this explicit provider supplies standard-onboarding intent, but the route remains unvalidated.

Install or start managed vLLM on a supported host, or explicitly enter the Deferred N1x preview.

NEMOCLAW_PROVIDER=install-vllm \
  nemoclaw onboard --non-interactive

On DGX Spark and DGX Station, NEMOCLAW_PROVIDER=install-vllm is sufficient for a non-interactive run. On N1x, that setting is explicit preview intent and waives only the pending physical-validation finding after the N1x identity qualifies. Add NEMOCLAW_EXPERIMENTAL=1 on a generic Linux NVIDIA GPU host. Non-interactive runs use the profile default unless you set NEMOCLAW_VLLM_MODEL.

The commands above invoke nemoclaw onboard directly, so a DGX Station run with no model or peer selects the deepseek-v4-flash profile default. Supplying NEMOCLAW_PROVIDER=install-vllm to the shell installer enters the Station host-preparation boundary while retaining that profile default. For paired non-interactive setup, follow Set Up vLLM on Two DGX Stations.

For a headless DGX Station setup that selects DeepSeek V4 Flash explicitly, use the environment-variable path instead of --station-deepseek.

NEMOCLAW_PROVIDER=install-vllm \
  NEMOCLAW_VLLM_MODEL=deepseek-v4-flash \
  nemoclaw onboard --non-interactive

For a headless or CI first install on a qualifying DGX Station after host preparation, set the same managed-vLLM recipe in the installer environment.

curl -fsSL https://www.nvidia.com/nemoclaw.sh | \
  NEMOCLAW_AGENT=openclaw \
  NEMOCLAW_NON_INTERACTIVE=1 \
  NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE=1 \
  NEMOCLAW_PROVIDER=install-vllm \
  NEMOCLAW_VLLM_MODEL=deepseek-v4-flash \
  NEMOCLAW_SANDBOX_NAME=my-assistant \
  bash

Select a Managed Model

Set NEMOCLAW_VLLM_MODEL=<slug> before onboarding to select a model without prompting. NemoClaw applies the registered vllm serve arguments, including the reasoning parser, tool-call parser, and --max-model-len.

Slug	Hugging Face model	Registered host profiles	Minimum compute capability	Notes
`qwen3.6-27b`	`Qwen/Qwen3.6-27B-FP8`	DGX Spark, DGX Station, Linux with an NVIDIA GPU	`8.9`	Supported override.
`qwen3.6-35b-a3b-nvfp4`	`nvidia/Qwen3.6-35B-A3B-NVFP4`	DGX Spark, N1x	`12.1`	DGX Spark and N1x default.
`muse-glimmer-30b`	`Inferact/Muse-Glimmer-30B-NVFP4-W4A4`	DGX Spark, Linux x86_64 with an NVIDIA GPU	`12.0` on Linux; `12.1` on DGX Spark	Experimental choice for text, reasoning, and structured tool calls. NemoClaw does not enable or support DFlash for this profile.
`nemotron-3.5-lightning-30b`	`nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4`	DGX Spark, Linux x86_64 with an NVIDIA GPU	`9.0` on Linux; `12.1` on DGX Spark	Experimental reasoning and tool-calling choice.
`nemotron-3-nano-4b`	`nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`	DGX Spark, DGX Station, Linux with an NVIDIA GPU	`8.9`	Generic Linux NVIDIA GPU default.
`deepseek-v4-flash`	`deepseek-ai/DeepSeek-V4-Flash`	DGX Station	`10.0`	DGX Station profile default outside express install.
`nemotron-3-ultra-550b-a55b`	`nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4`	DGX Station	`10.0`	DGX Station express-install selection with a pinned model revision and model-specific vLLM image.
`deepseek-r1-distill-70b`	`deepseek-ai/DeepSeek-R1-Distill-Llama-70B`	DGX Spark, DGX Station, Linux with an NVIDIA GPU	None	Gated and requires license acceptance.

Slugs are case-insensitive, and NemoClaw also accepts the full Hugging Face model ID or a registered served model name. An unrecognized value fails before image or model downloads and prints the valid slugs. A recognized override that does not support the detected host also fails before image or model downloads. The error names the selected model and detected host profile. A model whose quantized checkpoint needs a newer GPU than the host provides also fails before those downloads. Follow Check GPU Compute Capability for that check.

Gated models require a Hugging Face token and license acceptance. Follow Authenticate Hugging Face Downloads before onboarding so NemoClaw can forward the token temporarily to the one-shot model downloader.

export HF_TOKEN="<your-hf-token>"
NEMOCLAW_PROVIDER=install-vllm \
  NEMOCLAW_VLLM_MODEL=deepseek-r1-distill-70b \
  nemoclaw onboard --non-interactive

NemoClaw also accepts HUGGING_FACE_HUB_TOKEN as a compatibility alias. The host checks the token before docker pull, so a missing or empty token stops onboarding before bandwidth is spent on a request that would return 401.

Add Managed Serve Arguments

Set NEMOCLAW_VLLM_EXTRA_ARGS_JSON to a JSON array of individual non-blank vllm serve tokens for operator-owned options and overrides. NemoClaw trims and validates the array before downloads, shell-quotes each token, and appends the tokens after the registry defaults. When the array overrides --gpu-memory-utilization, NemoClaw validates the last value and uses that effective value for GPU-memory preflight, matching the final vllm serve command.

NEMOCLAW_PROVIDER=install-vllm \
  NEMOCLAW_VLLM_EXTRA_ARGS_JSON='["--max-num-seqs","2","--disable-log-requests"]' \
  nemoclaw onboard --non-interactive

If the selected image does not support an argument, the managed container exits and NemoClaw prints the log tail.

When the selected managed recipe defines a GPU-memory utilization value, NemoClaw checks nvidia-smi total and free memory for the Docker-selected GPU before large downloads. It repeats the same check with fresh telemetry immediately before launch so another workload cannot consume the required free GPU memory unnoticed during the download. For a qualified N1x or DGX Spark profile, if the selected GPU has a valid index and UUID but both memory values are exactly [N/A], NemoClaw warns that it cannot pre-validate utilization and continues without inferring available memory. Every other unavailable, missing, or malformed telemetry result stops onboarding. Insufficient numeric free memory also stops onboarding. Follow the diagnostic to restore valid telemetry, correct the selected GPU, or free the required memory. Then run nemoclaw onboard --resume.

To bound resource use while investigating long-context workflows on one DGX Spark, select the Qwen profile. For a Spark with at least 64 GB of detected unified memory, the following override disables async scheduling and lowers the context window, concurrent-sequence limit, and batch limit to the current N1x defaults.

NEMOCLAW_PROVIDER=install-vllm \
  NEMOCLAW_VLLM_MODEL=qwen3.6-35b-a3b-nvfp4 \
  NEMOCLAW_VLLM_EXTRA_ARGS_JSON='["--no-async-scheduling","--max-model-len","32768","--max-num-seqs","2","--max-num-batched-tokens","4096"]' \
  nemoclaw onboard --non-interactive

This override reduces request concurrency and context capacity, so it can reduce throughput. It does not guarantee protection against a DGX Spark host freeze.

Choose a Local Inference Server to compare vLLM with Ollama and NVIDIA NIM.
Set Up vLLM on Two DGX Sparks for the Experimental distributed DeepSeek V4 Flash profile.
Set Up vLLM on Two DGX Stations for the Deferred distributed-serving workflow.
Configure Inference Timeouts when model startup or validation needs more time.
Verify the Inference Route after setup.