Set Up Model Router

Model Router runs on the host and selects a model from a configured pool for each request. OpenShell registers it as an OpenAI-compatible provider while the sandbox remains on inference.local.

How the Router Fits

The Model Router option uses the routed inference profile in nemoclaw-blueprint/blueprint.yaml. During onboarding, NemoClaw starts the router proxy on host port 4000, waits for its health endpoint, and registers the nvidia-router provider with OpenShell. The sandbox does not call port 4000 directly. On a host without cached routing-model files, Model Router downloads and loads its routing model before the health endpoint responds. NemoClaw gives a running router up to 10 minutes to pass its health check and stops onboarding sooner if the process exits. NemoClaw tries to append router process output to ~/.nemoclaw/state/model-router.log. A nondefault gateway port uses ~/.nemoclaw/gateways/<port>/state/model-router.log. NemoClaw creates or reopens the log with owner-only permissions. Each Model Router start appends to the same file. NemoClaw does not truncate or rotate the file. If output capture fails, router startup continues without captured output. If the router does not pass its health check, NemoClaw tries one final health read. When that read returns an unhealthy-endpoint error, the startup error includes a redacted excerpt. When output capture succeeded, the startup error also includes the log path. When the log contains readable output from this run, the error includes a redacted excerpt. The owner-only log contains unredacted router output. Review the log before you share it. NemoClaw does not provide log-only cleanup. A successful uninstall stops the selected Model Router and removes its log with the selected gateway’s operational state. Review the uninstall scope before you use it for log removal.

Sandbox agent -> OpenShell -> Model Router on port 4000 -> NVIDIA API

Credentials flow through the OpenShell provider system. The sandbox never sees raw API keys.

Configure the Model Pool

The router model pool lives in nemoclaw-blueprint/router/pool-config.yaml. Edit that file to define the model names that the router can select.

The default pool routes between NVIDIA-hosted Nemotron models. It uses tolerance to select the lowest-cost model whose predicted quality stays within the configured threshold.

1
routing:
2
  method: prefill
3
  checkpoint: llm-router/checkpoints/prefill_router_qwen08b.pt
4
  tolerance: 0.20
5
  encoder: Qwen/Qwen3.5-0.8B
6
7
models:
8
  - name: nano
9
    litellm_model: "openai/nvidia/Nemotron-3-Nano-30B-A3B"
10
    cost_per_m_input_tokens: 0.05
11
    api_base: "https://integrate.api.nvidia.com"
12
13
  - name: super
14
    litellm_model: "openai/nvidia/nemotron-3-super-120b-a12b"
15
    cost_per_m_input_tokens: 0.10
16
    api_base: "https://integrate.api.nvidia.com"

Use these values to tune the accuracy and cost tradeoff.

Meet the Python Requirement

NemoClaw creates a host-side virtual environment for Model Router during onboarding. It probes python3.13, python3.12, python3.11, python3.10, and python3 in that order. The selected interpreter must have a version in the range [3.10, 3.14) and must import ensurepip, pyexpat, ssl, and venv without error.

Before creating the environment, NemoClaw checks for at least 3 GiB of free or reclaimable capacity on its filesystem. This capacity covers the installed Python packages and download staging. When NemoClaw replaces a NemoClaw-managed environment, storage that removal can reclaim counts toward the requirement. If capacity is below the requirement, onboarding stops before creating the environment and reports how much space to free. Free the reported capacity, then run nemohermes onboard --resume. If the filesystem does not report capacity, onboarding prints the reason and continues.

If no candidate qualifies, onboarding stops and prints the failure for each candidate. Set NEMOCLAW_MODEL_ROUTER_PYTHON to an absolute path to probe only one interpreter.

$
NEMOCLAW_MODEL_ROUTER_PYTHON=/opt/homebrew/bin/python3.12 nemohermes onboard

The pin is strict. NemoClaw rejects relative command names and stops with the failure reason if the selected interpreter does not qualify.

Onboard Non-Interactively

Set the routed provider and NVIDIA credential before running onboarding.

$
NEMOCLAW_PROVIDER=routed NVIDIA_INFERENCE_API_KEY=<your-key> nemohermes onboard --non-interactive

Related Topics

• About Inference Routing explains the host and sandbox boundary.
• View the Active Inference Route shows the active provider and model.