InferenceManage Inference

Configure Model Limits

View as Markdown

Configure explicit model limits before onboarding so NemoClaw can bake them into the sandbox image. Changing an explicit build-time model limit on an existing sandbox requires fresh recreation.

Set OpenClaw Limits

OpenClaw accepts an explicit context window and maximum output-token count.

VariableValuesDefault
NEMOCLAW_CONTEXT_WINDOWPositive integer in tokens131072
NEMOCLAW_MAX_TOKENSPositive integer in tokens4096

Export one or both values before onboarding.

export NEMOCLAW_CONTEXT_WINDOW=65536
export NEMOCLAW_MAX_TOKENS=8192
nemoclaw onboard

NemoClaw ignores invalid values and uses the default instead.

Use Detected Local Limits

When NEMOCLAW_CONTEXT_WINDOW is unset, NemoClaw can use a context length reported by the selected local server. Local Ollama reports the loaded model’s runtime context length. Local vLLM and OpenAI-compatible endpoints can report max_model_len through /v1/models. Local llama.cpp reports the selected model’s served context window as meta.n_ctx through the authenticated /v1/models response. NemoClaw does not use the training limit in meta.n_ctx_train because the server can use a smaller context window. NemoClaw does not adopt meta.n_ctx when it is missing, malformed, or outside its accepted range. For local llama.cpp, NemoClaw warns before it ignores an invalid NEMOCLAW_CONTEXT_WINDOW. It uses a valid detected value instead or leaves the value unset.

Set a valid NEMOCLAW_CONTEXT_WINDOW when you need to override the detected value.

Recreate an Existing Sandbox

Model limits are build-time settings. Recreate the named sandbox after changing a supported value.

nemoclaw onboard --fresh --name <sandbox-name> --recreate-sandbox