> This page is for NeMo Relay, version v0.10.0.
> For other versions, use one of these documentation indexes:
> - Latest (v0.10.0) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt
> - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt
> - v0.10.0: https://docs.nvidia.com/nemo/relay/v0.10.0/llms.txt
> - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt
> - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt
> - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt
> - v0.6.0: https://docs.nvidia.com/nemo/relay/v0.6.0/llms.txt
> - v0.5.0: https://docs.nvidia.com/nemo/relay/v0.5.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/relay/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/relay/_mcp/server.

# Architecture and Startup

> Learn how coding harnesses, MCP clients, workers, and the daemon communicate.

On narrow screens, scroll diagrams horizontally to read every label.

## Process Roles

The harness launches `nemo-relay daemon mcp` as a Model Context Protocol (MCP)
process over standard input and output. This MCP connection keeps a reference
open with the daemon. It provides no tools. The daemon's **broker** decides
whether the user-machine route needs a new worker or can share an existing one.

The MCP process starts a worker on the **client computer** only when the broker
instructs it to. The worker loads the administrator's system configuration and
plugins. Install the daemon as a service. The harness starts MCP, and the broker
controls worker startup and shutdown.

The diagram shows how these processes communicate:

```mermaid
flowchart TD
    H["Coding harness<br />Codex, Claude Code, or Pi"]
    M["MCP lifecycle client"]
    F["Short-lived hook forwarder"]
    D["Daemon and route broker"]
    W["Worker for one machine-user"]
    P["Model provider"]
    H -. "launch and MCP stdio" .-> M
    H -. "launch with hook payload" .-> F
    M <-->|"WebSocket lifecycle and control"| D
    M -. "launch with protected grant" .-> W
    W <-->|"WebSocket lifecycle and control"| D
    H -->|"model requests"| D
    F -->|"hook requests"| D
    D -->|"authenticated requests"| W
    W -->|"model requests"| P
    D -->|"model requests in pass-through"| P
```

Dotted arrows show process startup. Solid arrows show communication. Responses
return along the request path. Model streams travel back through the worker and
daemon to the harness. Hook responses return through the hook forwarder.
The harness and hook forwarder do not contact a worker directly.

MCP and worker processes each open a persistent WebSocket control session to the
daemon. The daemon pushes route decisions and drain commands on those sessions.
Model and hook traffic use separate HTTP connections.

The daemon also exposes read-only HTTP operations for process health and
last-known worker assignments. `/healthz` describes the daemon process and its
global deployment mode. `/_nemo_relay/v1/workers`,
`/_nemo_relay/v1/workers/status`, and the per-worker status path read the
broker's in-memory state without probing workers. Routes without an assigned
worker, including pass-through routes, do not appear in worker results.

## Local Deployment

The diagram shows the processes on one computer:

```mermaid
flowchart TD
    subgraph PC["One computer"]
        H["Harness and hook forwarder"]
        M["MCP client"]
        D["Daemon service<br />127.0.0.1:47632"]
        W["Per-user worker<br />127.0.0.1:auto port"]
        H -->|"HTTP requests"| D
        H -. "launch" .-> M
        M <-->|"WebSocket control"| D
        M -. "launch" .-> W
        W <-->|"WebSocket control"| D
        D -->|"HTTP requests"| W
    end
    W -->|"HTTPS"| P["Model provider"]
    D -->|"HTTPS in pass-through"| P
```

A system daemon has its own service identity. Each user's MCP and worker use that
user's identity. Several harness sessions for the same identity share one worker.
Different users do not share runtime state. The operating system assigns the
default worker port. MCP starts workers when the broker requests them.

## Initialization Sequence

The diagram shows startup with a daemon that supports parallel initialization:

```mermaid
sequenceDiagram
    participant H as Harness
    participant M as MCP client
    participant D as Daemon
    participant W as Worker
    H->>M: Start with route credential in environment
    M->>D: Request challenge and verify daemon identity
    D-->>M: Signed challenge
    M->>D: Signed registration and credential binding
    H->>M: MCP initialize
    M-->>H: MCP initialized after authenticated registration
    Note over M,D: Worker starts in the background. Requests use passthrough until ready.
    alt Route needs a worker
        D-->>M: LaunchWorker and one-time grant
        M->>W: Launch with grant through protected stdin
        W->>D: Prove identity and grant, register endpoint
        W->>D: Ready over control socket
        D->>W: Probe HTTP readiness endpoint
        W-->>D: Readiness confirmed
        D-->>M: Route ready
    else Another MCP is starting or draining a worker
        D-->>M: WaitForWorker
        D-->>M: Push next broker decision
    else Worker is already ready
        D-->>M: ReuseWorker
    else Pass-through route
        D-->>M: UsePassThrough
    end
    H->>D: Model or hook request with route credential
    Note over M,D: MCP control socket stays open for the session
```

After registration and authentication, MCP handles standard input and output
while the worker starts. MCP readiness alone does not prove that a worker is
running. New requests use passthrough during startup by default.

The daemon starts routing new requests to a worker after these checks pass:

* The worker has initialized its runtime and plugins.
* Its control connection is authenticated.
* Its HTTP readiness probe succeeds.

Use `--require-worker` to return HTTP `503` while these checks are pending.
With older daemons, MCP waits for a usable route before it handles initialization.

## Shutdown and Recovery

The diagram shows the main worker states and how they change:

```mermaid
stateDiagram-v2
    [*] --> Empty
    Empty --> Activating: First MCP reference
    Activating --> Ready: Worker ready
    Activating --> PassThrough: Activation fails
    Ready --> Recovering: Worker fails
    Recovering --> Activating: One MCP relaunches
    Ready --> Draining: Last MCP leaves
    PassThrough --> Activating: Connected MCP retries after failure
    PassThrough --> Empty: Last MCP leaves
    Draining --> Empty: Worker exits
```

The `--pass-through` flag disables worker startup. A draining worker cannot
return to service. Its replacement waits for it to exit. During that wait, new
requests use passthrough by default. Accepted requests have up to two minutes to
finish. Workers stop accepting new requests if they lose authenticated daemon
control.

Refer to [Identity and Lifecycle](/daemon/reference#understand-broker-identity-and-lifecycle)
for reconnect grace, remote keepalive timing, restart recovery, and all broker states.

Worker startup has no deadline. The daemon logs a progress message every
60 seconds and records how long startup took when the worker joins the route.
If startup fails, the daemon retries after one second, then two, four, and so on.
Retry delays stop increasing at 60 seconds. There is no attempt limit.

If the MCP client that starts the worker disconnects, the daemon cancels its
launch grant. Another connected MCP can then receive a new grant.

When startup fails or the daemon cancels a launch, it releases the worker's
staged registration right away. Published workers keep their reconnect grace
period. Slow writes to daemon state do not block request streams or health checks.

Each request keeps the destination selected when it starts. Streams already
sent through passthrough finish on that path. Later requests use the ready
worker. Relay never sends the same submitted request to both destinations.