InferenceLocal Inference

Choose a Local Inference Server

View as Markdown

NemoClaw supports Ollama, vLLM, and NVIDIA NIM as local inference servers. Choose the option that matches your host, model, and operational needs.

The agent inside the sandbox sends inference traffic to inference.local. OpenShell intercepts that traffic and forwards it to the local endpoint configured during onboarding.

Compare the Options

OptionWhen to use itAvailabilityRuntime API
OllamaYou want the default local option and want NemoClaw to install, start, or use Ollama on supported hosts.Appears when Ollama is installed or running, and the wizard can offer installation on supported hosts.Ollama through the managed local route.
Existing vLLMYou already run vLLM on localhost:8000.Appears when NemoClaw detects the server./v1/chat/completions.
Managed vLLMYou want NemoClaw to pull an image, download model weights, and manage the server container.Appears by default on DGX Spark and DGX Station, while generic Linux NVIDIA GPU hosts require NEMOCLAW_EXPERIMENTAL=1 or NEMOCLAW_PROVIDER=install-vllm./v1/chat/completions.
NVIDIA NIMYou want NemoClaw to pull and manage a validated NIM container on a NIM-capable NVIDIA GPU.Experimental and requires NEMOCLAW_EXPERIMENTAL=1./v1/chat/completions.

Ollama selects among installed or starter model tags and validates the selected model. Managed vLLM uses host-specific model profiles and lets you select a supported registry model. NVIDIA NIM filters its available models by detected GPU VRAM.

Choose Ollama

Choose Ollama when you want the default local setup path. The wizard can detect a running daemon, install or upgrade Ollama on supported macOS and Linux hosts, and work with Windows-host Ollama from WSL when Docker Desktop integration is available.

Some model and template combinations can return tool calls as plain text under realistic agent load. OpenClaw onboarding validates structured tool calls and stops when the selected model does not provide the required behavior.

Refer to Set Up Ollama.

Choose vLLM

Choose vLLM when you already operate a compatible server or want a managed container on a supported NVIDIA GPU host. NemoClaw forces the Chat Completions API path because the vLLM Responses endpoint does not run the configured tool-call parser.

Refer to Set Up vLLM.

Choose NVIDIA NIM

Choose NVIDIA NIM when you want a managed NIM container and your host has a NIM-capable NVIDIA GPU. The path is experimental, requires NGC registry access, and can fail when a selected image does not publish a manifest for the host architecture.

Refer to Set Up NVIDIA NIM.

Use Another Server

Use a custom endpoint when your server is not one of the managed local options. NemoClaw supports servers that expose an OpenAI-compatible API and supports compatible Anthropic routes with agent-specific runtime requirements.