InferenceManage Inference

Configure Model Limits

View as Markdown

Configure explicit model limits before onboarding so NemoClaw can bake them into the sandbox image. Changing an explicit build-time model limit on an existing sandbox requires fresh recreation.

Set OpenClaw Limits

Set the Hermes Context Window

Hermes accepts NEMOCLAW_CONTEXT_WINDOW as its model-limit override.

VariableValuesDefault
NEMOCLAW_CONTEXT_WINDOWPositive integer, at least 64000 tokensUnset so Hermes auto-detects
$export NEMOCLAW_CONTEXT_WINDOW=65536
$nemohermes onboard

When onboarding resolves a valid value, NemoClaw writes it as model.context_length in /sandbox/.hermes/config.yaml. For non-Ollama endpoints, the field remains unset when no explicit or probed value is available so Hermes can auto-detect it from the endpoint. During inference set, NemoClaw recomputes the context window for the target model. It writes model.context_length when it resolves a value and omits the field when Hermes must use endpoint auto-discovery. When NemoClaw starts Local Ollama on macOS or Linux, it requests at least 64000 tokens. Fresh onboarding then verifies the loaded model’s actual context_length through /api/ps. Resumed onboarding and sandbox rebuilds warm the exact recorded Ollama model and repeat this verification before reusing its route. When NEMOCLAW_CONTEXT_WINDOW is larger than 64000, the Ollama runtime must provide at least that larger value. Onboarding stops before building the sandbox when Ollama reports a smaller, missing, or malformed value and shows the required OLLAMA_CONTEXT_LENGTH value for restarting Ollama. Setting NEMOCLAW_CONTEXT_WINDOW does not raise the Ollama daemon’s runtime context or bypass this check.

Use Detected Local Limits

When NEMOCLAW_CONTEXT_WINDOW is unset, NemoClaw can use a context length reported by the selected local server. Local Ollama reports the loaded model’s runtime context length. Local vLLM and OpenAI-compatible endpoints can report max_model_len through /v1/models.

Set NEMOCLAW_CONTEXT_WINDOW when you need to override the detected value.

Recreate an Existing Sandbox

Model limits are build-time settings. Recreate the named sandbox after changing a supported value.

$nemohermes onboard --fresh --name <sandbox-name> --recreate-sandbox