Set Up llama.cpp | NVIDIA NemoClaw

NemoClaw provides two first-class llama.cpp paths. You can attach an authenticated server that you operate, or let NemoClaw select and materialize a compatible repository-owned profile. Both paths are experimental and use OpenAI Chat Completions.

New Deep Agents Code and Hermes sandboxes use the recorded local endpoint through a sandbox-attached OpenShell provider. OpenShell supplies the credential at egress; the sandbox receives only a credential reference. Refer to How Inference Routing Works for the qualification and existing-sandbox boundaries.

Choose a llama.cpp Path

Path	NemoClaw manages	You manage	Use it when
Existing server	Provider registration, route validation, and sandbox inference through OpenShell.	The server binary or container, GGUF file, launch configuration, credential, upgrades, and lifecycle.	Your authenticated server already satisfies the llama.cpp fingerprint contract on loopback port `8081`.
Managed NVIDIA GPU host	The exact YAML-selected image, GGUF acquisition and verification, launch configuration, credential, Docker lifecycle, diagnostics, and cleanup.	A qualified DGX Spark Arm64, Linux x86_64 NVIDIA GPU, or Windows WSL N1x or Station GB300 host, plus the recipe readiness requirements and any required Hugging Face credential.	You want a repository-owned managed llama.cpp profile without managing `llama-server` directly.

Compared with other local options, Ollama emphasizes a simple local model workflow, while managed vLLM provides host-specific NVIDIA GPU profiles and broader model selection. NVIDIA NIM provides NVIDIA-packaged inference containers for validated NIM-capable GPUs. Managed llama.cpp instead runs a repository-owned GGUF recipe with one request slot and no general launch-flag overrides. For a managed runtime, each fingerprint request can run for up to 30 seconds within a shared 90-second fingerprint deadline during onboarding, status, and context-window checks. An operator-managed server retains the five-second request limit. Refer to Choose a Local Inference Server for the complete comparison.

Attach an Existing llama.cpp Server

Use this path when you operate llama-server and want NemoClaw to register its authenticated endpoint without owning its process or model. The server must satisfy all of these requirements:

Listen over HTTP on 127.0.0.1:8081 and expose its OpenAI-compatible API under /v1.
Make that authenticated API reachable from the sandbox network at host.openshell.internal:8081.
Require the same native bearer key supplied through NEMOCLAW_LLAMACPP_LOCAL_TOKEN.
Expose bounded native llama.cpp evidence through /v1/models, /health, /props, and either /metrics or the native metrics-not-supported response.
Report a stable, non-path served model alias.
Return native llama.cpp model metadata without conflicting model entries.

On native Docker, publishing only -p 127.0.0.1:8081:8081 can leave the sandbox route unreachable. Keep the loopback publication and also publish -p <Docker-gateway-IP>:8081:8081 for the OpenShell network. For a host process, configure its listener or forwarding so both addresses work. Retain native API-key authentication and restrict non-loopback access to the sandbox network. Onboarding stops when the existing network probe confirms that the attached server cannot be reached.

Configure the server with its own supported mechanism before you run NemoClaw. Start the server with an explicit served model alias. A server that starts without an alias reports its model file path as the model ID. NemoClaw rejects a path as a served model alias. Enable metrics when available, and use an API-key file instead of placing the key in process arguments when your llama.cpp build supports those options. If the server exposes multiple models, identify the served alias through NEMOCLAW_MODEL.

NemoClaw takes the served model alias from the /v1/models entry that it selects. It also compares that alias with model_alias in /props when /props returns that field. NemoClaw refuses the attachment when the two values differ. If /props omits model_alias, NemoClaw attaches the server using the remaining native evidence.

For interactive onboarding, run:

nemo-deepagents onboard

Select Local llama.cpp and provide the native API key. Interactive attachment succeeds only when the server exposes exactly one native model. If the server exposes multiple models, use non-interactive onboarding and set NEMOCLAW_MODEL to the served alias.

For non-interactive onboarding, export NEMOCLAW_LLAMACPP_LOCAL_TOKEN from your secret manager and run:

: "${NEMOCLAW_LLAMACPP_LOCAL_TOKEN:?Export the native llama.cpp API key first}"
NEMOCLAW_PROVIDER=llama-cpp \
NEMOCLAW_MODEL="<served-model-alias>" \
NEMOCLAW_SANDBOX_NAME=my-assistant \
  nemo-deepagents onboard --non-interactive --yes-i-accept-third-party-software

NemoClaw reads NEMOCLAW_LLAMACPP_LOCAL_TOKEN from the environment for this command. You can omit NEMOCLAW_MODEL only when the server exposes exactly one native model. It refuses unauthenticated, ambiguous, non-native, conflicting, or differently addressed servers instead of guessing their identity. It does not start, stop, upgrade, or remove an attached server.

Verify the registered route:

nemo-deepagents my-assistant status
nemo-deepagents my-assistant doctor

If a compatible server does not satisfy the first-class fingerprint, use Set Up an OpenAI-Compatible Endpoint. That path supports operator-selected endpoints and ports without claiming llama.cpp-specific identity or lifecycle ownership.

Install Managed llama.cpp on an NVIDIA GPU Host

Use this experimental path when you want NemoClaw to manage one declarative llama.cpp recipe on a qualified DGX Spark Arm64, Linux x86_64 NVIDIA GPU, or Windows WSL N1x or Station GB300 host. On DGX Spark, the recommended recipe serves NVIDIA Nemotron 3 Nano 30B-A3B through the OpenAI Chat Completions API. The Meta Muse Glimmer 30B recipe remains available by explicit selection, and existing Muse-owned installations are not migrated. Windows WSL N1x and Station GB300 Express leave provider selection to onboarding. Accepting Windows Express and its third-party software notice is the explicit opt-in to this Experimental surface. During Windows Express, onboarding automatically selects the Qwen 3.6 35B-A3B recipe after the applicable N1x or Station GB300 readiness requirements pass.

This path is an experimental implementation. It does not establish a supported agent, model, and runtime tuple until the protected qualification and activation gates pass. The managed image must contain native llama-server and curl. The model container has no published ports and remains on an internal Docker network. NemoClaw uses its authenticated host bridge and requires successful host-loopback and OpenShell route checks before activation. The protected qualification runner’s temporary registry still requires Docker Engine 28.3.3 or newer with its default protected NAT and firewall behavior. The Muse Glimmer profile remains Experimental after bounded physical DGX Spark qualification of text, separated reasoning, and one structured tool call.

Before you start, confirm these prerequisites:

Use a qualified DGX Spark Arm64 host, a Linux x86_64 host with one NVIDIA GPU, or a qualifying Windows WSL N1x or Station GB300 host.
Use NVIDIA driver version 580.65.06 or later on each managed host profile.
Confirm that Docker is operational and satisfies the selected profile’s readiness checks.
Confirm that the NVIDIA Container Toolkit is operational.
Confirm that NVIDIA Container Device Interface support is healthy.
Stop any process that uses host port 8081.
Allow capacity for the pinned images, the GGUF file, and same-filesystem download staging.
Export HF_TOKEN only when the YAML-declared Hugging Face source requires authentication.

For Windows WSL N1x, also confirm these requirements:

Use Arm64 WSL with one GPU whose normalized identity is NVIDIA RTX Spark N1X, NVIDIA RTX Spark N1X (5120-core Blackwell RTX GPU), or NVIDIA RTX Spark N1X (6144-core Blackwell RTX GPU).
Use the default local Docker Desktop context with at least 48,000 MiB of Docker memory.
Unset DOCKER_HOST and select Docker’s default context before you select the managed recipe.
Provide at least 48,000 MiB of GPU memory.
Pass Docker storage, runtime, NVIDIA GPU integration, and Docker Desktop GPU passthrough readiness.

For Windows WSL Station GB300, also confirm these requirements:

Use Arm64 WSL on a Windows product identified as Station GB300.
Require the Docker Desktop proof to reconcile every reported GPU and identify exactly one proved NVIDIA GB300 GPU UUID.
Use the default local Docker Desktop context with at least 48,000 MiB of Docker memory.
Unset DOCKER_HOST and select Docker’s default context before you select the managed recipe.
Provide at least 48,000 MiB of aggregate GPU memory.
Pass Docker storage, runtime, NVIDIA GPU integration, and Docker Desktop GPU passthrough readiness.

If neither readiness contract matches, Windows Express selects WSL-local Ollama during onboarding. If a required readiness check fails after selection starts, onboarding stops before installation. Managed WSL selection rejects any other Docker selector.

NemoClaw uses HF_TOKEN only while it acquires the model. It does not write the value to managed llama.cpp state, the receipt, the serving container environment, or the sandbox registry. Run unset HF_TOKEN after onboarding when no other process needs it.

When the artifacts are not already present, onboarding pulls digest-pinned images from external registries and acquires the YAML-declared GGUF file through the standard Hugging Face cache. It reuses locally present pinned images and a cached GGUF only after NemoClaw verifies their identities. It creates an owner-only API key, a host-loopback listener on port 8081, and a Docker internal network. Only one managed llama.cpp runtime can exist per Docker authority, regardless of the owning OpenShell gateway or sandbox.

Run interactive onboarding on the qualified host:

nemo-deepagents onboard

NemoClaw lists compatible managed llama.cpp profiles with automatic selection in descending YAML priority order. During interactive onboarding without an explicit provider request, the menu ignores NEMOCLAW_LLAMACPP_RECIPE and NEMOCLAW_SERVING_PRESET and marks the unique highest-priority compatible profile as (recommended). On DGX Spark, the recommended profile appears as Managed llama.cpp: NVIDIA Nemotron 3 Nano 30B-A3B on one DGX Spark (recommended). On Linux x86_64, the recommended profile identifies one NVIDIA GPU instead. During Windows Express or non-interactive onboarding, a qualifying Windows WSL N1x or Station GB300 host selects managed llama.cpp without a provider prompt when no provider is requested or recorded and NEMOCLAW_MODEL is unset. With NEMOCLAW_LLAMACPP_RECIPE unset, it selects the compatible Qwen 3.6 35B-A3B managed profile for that host; a compatible explicit recipe selects the same managed recipe. Ordinary interactive onboarding lists the same recommended profile but allows another provider selection. The explicit-only Meta Muse Glimmer profile does not appear in the menu. Select any listed profile to install its exact repository-owned recipe. The selected menu entry determines the exact recipe even when NEMOCLAW_LLAMACPP_RECIPE names another recipe.

Use --profile to select a profile by its stable ID, including the explicit-only Meta Muse Glimmer profile. Confirm that nemo-deepagents profiles list reports the profile as compatible first:

nemo-deepagents profiles list
nemo-deepagents onboard --profile llama-cpp.dgx-spark-gb10.single.muse-glimmer-30b

The profile selects the managed llama.cpp provider and installs the recipe that the profile declares. Do not combine --profile with NEMOCLAW_PROVIDER, NEMOCLAW_MODEL, or NEMOCLAW_LLAMACPP_RECIPE.

For non-interactive onboarding without --profile, select the repository-owned recipe explicitly:

NEMOCLAW_PROVIDER=install-llama-cpp \
NEMOCLAW_LLAMACPP_RECIPE=llama-cpp.nemotron-3-nano-30b-a3b.spark-single.v1 \
NEMOCLAW_SANDBOX_NAME=my-assistant \
  nemo-deepagents onboard --non-interactive --yes-i-accept-third-party-software

Use llama-cpp.muse-glimmer-30b.spark-single.v1 to select the Meta Muse Glimmer recipe explicitly. For Windows Express or non-interactive onboarding on a qualifying Windows WSL N1x or Station GB300 host, leave NEMOCLAW_PROVIDER and NEMOCLAW_MODEL unset for automatic managed-provider selection. Leave NEMOCLAW_LLAMACPP_RECIPE unset for the automatic Qwen recipe, or set NEMOCLAW_LLAMACPP_RECIPE=llama-cpp.qwen3-6-35b-a3b.n1x-wsl.v1 when automation must make the selected recipe explicit.

For non-interactive onboarding or an explicit install-llama-cpp provider request, an unset NEMOCLAW_LLAMACPP_RECIPE selects the unique highest-priority compatible automatic profile. If more than one compatible automatic profile has the highest priority, automatic non-interactive selection stops instead of selecting by catalog order. Interactive onboarding lists the tied profiles without a recommendation so you can select a recipe. For non-interactive onboarding or an explicit provider request, set NEMOCLAW_LLAMACPP_RECIPE to a recipe ID to select a compatible lower-priority profile or make the selection visible in automation. An unknown recipe, an incompatible host, or more than one compatible profile for the requested recipe stops onboarding before installation effects. Do not set NEMOCLAW_MODEL for this path. The repository YAML recipe is authoritative for the model file, revision, digest, image, launch settings, resource limits, readiness probes, and disabled surfaces. NemoClaw does not provide hidden model, port, image, or launch-flag overrides for the managed path.

The shipped NVIDIA Nemotron recipe continues to use the nemotron-v3-embedded chat template. A repository recipe that uses an image-owned Jinja template must declare this typed contract:

serve:
  chatTemplate: container-jinja-file
  chatTemplateFile: /usr/local/share/nemoclaw/llama-cpp/chat-templates/model-canonical.jinja
  reasoning:
    format: deepseek
    mode: auto

The template path must stay directly under /usr/local/share/nemoclaw/llama-cpp/chat-templates/. Its base name must start with an alphanumeric character and contain at most 128 alphanumeric, ., _, or - characters before the .jinja suffix. Subdirectories, path traversal, and host paths are invalid. The optional reasoning declaration supports only format: deepseek with mode: auto. The nemotron-v3-embedded contract rejects both chatTemplateFile and reasoning; recipes cannot supply arbitrary template or reasoning flags.

The Muse Glimmer and Qwen 3.6 recipes use the Jinja chat template embedded in their GGUF files and pin low reasoning strength:

serve:
  chatTemplate: model-embedded-jinja
  chatTemplateArguments:
    reasoningStrength: low

The typed contract accepts low, medium, high, or xhigh, but the selected recipe fixes the value and does not expose an onboarding override.

Onboarding performs these actions:

Resolves the selected recipe and profile against a fresh host readiness report.
Reuses locally present pinned downloader, runtime, and probe images, and pulls only the missing digest-pinned images declared by the recipe.
Reuses a verified GGUF from the shared ~/.cache/huggingface/ cache, or acquires the file through the existing Hugging Face mechanism when it is absent or invalid.
Verifies the GGUF size and SHA-256 digest before it starts the runtime.
Runs the authenticated container as the current non-root user with one NVIDIA GPU and no CPU fallback.
Configures native llama-server with its API-key file, one serving slot, and the recipe’s context and socket-timeout settings.
Starts a host-owned private bridge on 127.0.0.1:8081 and connects the container to a Docker internal network without a Docker-published port.
Verifies host-loopback and OpenShell reachability before registering the provider.
Registers llama-cpp-local and configures the agent’s OpenShell inference route described above.

The runtime cannot download a model and has no egress on its internal Docker network. The recipe disables the Web UI, slot inspection, server tools, agent mode, Model Context Protocol proxy, router, and multimodal projection. The API key enters the container through an owner-only read-only file, not a process argument. Managed requests have a 1 MiB body limit, a header ceiling of 32 KiB, a 4,096-token output ceiling, and a 900-second request deadline. The host’s stricter HTTP header limit continues to apply when present. The recipe’s serve.limits.requestTimeoutSeconds configures native llama-server read/write timeouts. The bridge’s fixed 900-second budget applies to the total request. On cancellation, deadline expiry, or bridge shutdown, NemoClaw verifies that the owned remote request has stopped before releasing admission. If cleanup cannot be verified, the bridge closes its listeners and exits with an error; it does not admit another request. Correct the reported Docker or ownership failure, then rerun the same onboarding selection. The new bridge reconciles the exact owned requests before opening its listeners.

The Muse Glimmer profile enables text responses, reasoning, and structured tool calls. It does not load a multimodal projector or a DFlash draft model because multimodal projection and speculative decoding remain disabled.

Podman and Kubernetes do not provide the required host-local-inference capability for this path. Selecting either runtime fails before model acquisition or runtime mutation and never falls back to Docker.

Verify the Managed Runtime

Run the managed runtime, route, and host-port checks:

nemo-deepagents my-assistant status
nemo-deepagents my-assistant doctor
docker inspect --format '{{json .HostConfig.PortBindings}}' nemoclaw-llama-cpp
docker inspect --format '{{json .NetworkSettings.Ports}}' nemoclaw-llama-cpp

Accept the result when status reports Managed llama.cpp: running, the inference route is healthy, and doctor exits with status 0. HostConfig.PortBindings must be empty. NetworkSettings.Ports must contain no host bindings; 8081/tcp can appear with a null value when the image declares that port. The authenticated host bridge provides reachability without publishing the model container’s ports. These checks do not establish agent and model qualification.

status reports the recipe ID, model digest, image reference, endpoint, and lifecycle state without exposing the API key. doctor distinguishes identity, runtime, and route failures and provides a recovery hint. Refer to CLI Commands for complete command behavior.

Recover the Managed Runtime

During managed installation, NemoClaw first checks internal runtime readiness and host-loopback health. It repeats these checks when onboarding resumes. It then runs the authoritative OpenShell Docker bridge probe on fixed port 8081. On Docker Desktop in WSL, NemoClaw runs the loopback health probe from the WSL CLI process because a host-network container cannot reach the WSL loopback listener. If the bridge probe cannot connect or times out after the earlier checks pass, onboarding stops without changing UFW. On Docker Desktop in WSL, this failure reports the private sandbox bridge probe result. Before retrying onboarding, confirm these conditions:

Docker Desktop is running.
Its WSL integration is enabled for this distribution.
The OpenShell Docker network and host gateway are available.

The WSL path does not print a UFW command.

On native Linux, when the bridge reports a valid narrow subnet and a gateway IP address inside that subnet, the error reports these values:

The detected Docker network.
The source subnet.
The gateway IP address.
The fixed port 8081.
The narrow UFW command.

On native Linux, if the bridge topology is missing or invalid, onboarding fails closed without printing topology or a UFW command. Inspect the OpenShell Docker bridge configuration before you retry onboarding.

On native Linux, the printed command changes UFW. Confirm that the reported Docker network, source subnet, and gateway IP address belong to the OpenShell Docker bridge before you run it. The rule allows TCP traffic only from that subnet to port 8081 on that gateway IP address.

On native Linux, run the command from the onboarding error. It has this form:

sudo ufw allow from <subnet> to <gateway-ip> port 8081 proto tcp

On native Linux, after you apply the rule, rerun the same onboarding selection.

On native Linux, remove the rule if onboarding still reports the bridge failure or the managed runtime no longer needs it:

sudo ufw --force delete allow from <subnet> to <gateway-ip> port 8081 proto tcp

On native Linux, use the same subnet and gateway IP address that appeared in the original command.

If resumed onboarding reports that it cannot reserve host-local inference, run nemo-deepagents <sandbox-name> doctor. If the diagnostic identifies the former WSL Qwen image, resume is unavailable. Follow the exact owned-runtime removal and fresh onboarding procedure in Upgrade or Roll Back after a qualified native image is pinned. For other runtime or gateway-authority conditions, correct the reported condition before you rerun the same provider and recipe selection. NemoClaw resumes only the persisted Docker authority, image, recipe, model digest, network, and runtime identity. It reconciles an unfinished create journal before it starts a new runtime. Phases before receipt-prepared are rolled back and retired, while receipt-prepared is finalized.

For a current managed image, if status reports stopped or preparing, rerun the same onboarding selection to resume the runtime. If it reports absent, conflict, or unknown, inspect the reported identity or Docker-authority error before retrying. Do not remove a same-name container or network unless its labels and identifiers match the persisted ownership state.

The model remains in the shared Hugging Face cache during recovery. NemoClaw reuses it only after the YAML-declared revision, file, size, digest, and filesystem identity verify. Refer to Host Files and State before changing managed state or cached artifacts.

Upgrade or Roll Back

NemoClaw does not independently upgrade an attached llama.cpp server. Upgrade or roll back that server and its model through the operator-owned process, then rerun onboarding so NemoClaw validates the endpoint again.

The managed path does not provide an in-place model, image, recipe upgrade, downgrade, or rollback command. The installed runtime remains bound to the repository YAML authority recorded during onboarding. Do not edit the ownership, receipt, or recipe-digest state to force a migration.

If an updated NemoClaw release changes the recorded authority, resume fails closed and preserves the existing state.

An existing Windows WSL Qwen runtime with the former pinned image remains bound to its recorded recipe, receipt, and journal. NemoClaw can remove that runtime through the existing journal-aware cleanup owner without replacing its image. The former image cannot start or resume. Other unrecognized authority changes fail closed and preserve the existing state.

Before destroying the sandbox, save any sandbox files you need. destroy removes the sandbox, its owned runtime, and its network but preserves the shared model cache. After a qualified native image is pinned, run nemo-deepagents <sandbox-name> destroy. Confirm that the sandbox no longer appears in nemo-deepagents list. Then run onboarding again.

For an earlier Station GB300 runtime with no recorded GPU UUID, NemoClaw also refuses startup because the old one-GPU request can select another card. After a qualified GB300 image is pinned in the recipe, save any sandbox files you need, run nemo-deepagents <sandbox-name> destroy --yes, confirm the sandbox is absent with nemo-deepagents list, and onboard again. The destroy command verifies and removes the exact owned runtime; do not edit the owner file or remove a same-name Docker resource by hand. Other managed llama.cpp recipe changes need release-specific migration guidance.

Remove the Managed Runtime

Run:

nemo-deepagents my-assistant destroy

Although the setup steps above target DGX Spark, this removal and recovery behavior also applies to other experimental managed llama.cpp profiles on Linux. NemoClaw qualifies the Docker operation authority recorded during managed llama.cpp onboarding. It completes this qualification before it requests sandbox deletion or reconciles an already-absent sandbox. This authority includes Docker endpoint selection and identity, Docker executable identity, and command environment. When cleanup relies on private managed llama.cpp state, NemoClaw proves Docker availability and the container and network before OpenShell sandbox deletion. It retains the identity of that private lifecycle state across sandbox deletion and rejects cleanup if the identity changes. If any pre-delete proof fails, NemoClaw reports the error and stops the destroy operation.

If the reported Docker operation authority mismatch involves endpoint selection, inspect the current Docker configuration and selectors:

docker context show
printf 'DOCKER_CONFIG=%s\n' "${DOCKER_CONFIG:-<unset>}"
printf 'DOCKER_CONTEXT=%s\n' "${DOCKER_CONTEXT:-<unset>}"
printf 'DOCKER_HOST=%s\n' "${DOCKER_HOST:-<unset>}"
printf 'DOCKER_TLS=%s\n' "${DOCKER_TLS:-<unset>}"
printf 'DOCKER_TLS_VERIFY=%s\n' "${DOCKER_TLS_VERIFY:-<unset>}"
printf 'DOCKER_CERT_PATH=%s\n' "${DOCKER_CERT_PATH:-<unset>}"

Compare this output with the shell or automation that ran onboarding. Restore the configuration and selection used during onboarding with the matching path:

If onboarding used a custom DOCKER_CONFIG directory, restore it first:
```
export DOCKER_CONFIG="<onboarding-docker-config>"
```
If onboarding used Docker’s default configuration directory, unset DOCKER_CONFIG.
If onboarding used DOCKER_CONTEXT or Docker’s persisted current context, select the original context explicitly:
```
unset DOCKER_HOST
export DOCKER_CONTEXT="<onboarding-context>"
```

If onboarding used DOCKER_HOST, restore that selection:

unset DOCKER_CONTEXT
export DOCKER_HOST="<onboarding-docker-host>"

For either selection path, restore DOCKER_TLS, DOCKER_TLS_VERIFY, and DOCKER_CERT_PATH to their onboarding state. Unset each TLS variable that was unset or empty during onboarding. These variables are part of the endpoint-selection tuple for both Docker contexts and DOCKER_HOST.

After restoring the recorded Docker configuration and selector, run docker info. Confirm that it reports the Docker daemon used during onboarding. Rerun the earlier destroy command.

NemoClaw stores opaque authority identity and binding data, not the original Docker executable or command environment values. For any other authority mismatch, do not guess Docker selector values. Retry the earlier destroy command from the same shell or automation environment that ran onboarding. After OpenShell confirms sandbox deletion, the command removes the managed llama.cpp container, internal network, API key, and gateway-scoped ownership state. It preserves ~/.cache/huggingface/ because other applications can use that cache. If cleanup fails, NemoClaw preserves the ownership state and any existing sandbox registry entry for a retry.

After destroy succeeds, run:

nemo-deepagents list

Confirm that my-assistant no longer appears.

For full NemoClaw removal, run nemo-deepagents uninstall. Full uninstall applies the ownership checks and preserves the shared Hugging Face cache by default. Pass --delete-models only when you accept deletion of all non-credential data in the current user’s shared cache after managed model runtimes stop. The same flag also deletes every model installed in the host’s local Ollama inventory. Uninstall preserves the Hugging Face token and stored_tokens authentication files.

Choose a Local Inference Server to compare llama.cpp with Ollama, vLLM, and NVIDIA NIM.
Verify the Inference Route after onboarding.
Configure Inference Timeouts for slow local models.