> This page is for version 0.2.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.server_runtime.request_handling.session_routing

Route each session's requests to the uvicorn worker that holds its state.

uvicorn's workers share one listening socket (unless `uvicorn_reuse_port` gives each its own),
so a request reaches whichever worker accepts the connection,
while a session's state lives in the memory of its current owner.
With more than one worker, each worker also serves the same app on a private Unix socket,
in a directory the main process creates for the server, and requests are routed by one rule:

1. Find the request's session ID.
2. Look up the session's owner, the worker that currently holds the session.
3. Forward the request to that worker's private socket, unchanged, and stream its reply back unchanged.
   If this worker owns the session, or the session has no owner, the request is handled here.

A request that arrives over a private socket is always handled where it arrives, so it is forwarded at most once.
If the owner's socket is gone, because its worker exited, the request gets a 410
instead of silently running against empty state.

Where the session ID comes from:

| Request                                                                              | Session ID                                                                |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| Any request to a resources server                                                    | The session cookie's `session_id`                                         |
| A resources tool call over MCP, or a request with an MCP session token and no cookie | The token's `sid`                                                         |
| An episode session's resources `/seed_session` or `/close_session`                   | The body's `resources_session_id`                                         |
| Any request to an agent server                                                       | The session cookie's `agent_session_id`                                   |
| An agent session seed or close                                                       | The body's `agent_session_id`                                             |
| An agent rollout stop without a cookie                                               | The path's `/rollouts/<rollout id>`, which the session's worker also owns |

The body is read only on the seed and close routes, where it names the session before any cookie exists.
If the cookie and the body name different sessions, the reply is 409.
An episode's resources session uses its `resources_session_id` as its cookie's `session_id` too,
so the seed, its close and every request in between name the session by the same ID.
A resources request with no session ID gets a new ID, which this router mints.
A session start, such as a seed, takes ownership of the session before its handler runs,
since its reply's cookie names the session.
Any other request takes ownership only if the handler used the session, before the reply leaves the worker.
A request that keeps nothing, such as a health check or `/aggregate_metrics`, takes no ownership and gets no cookie,
so no cookie names a session nobody owns.
A request whose cookie names a session whose ownership ended makes the worker that serves it the owner the same way.

Ownership is recorded in an owner file in the socket directory,
named by a hash of the session ID and holding the owner's worker ID.
The worker that creates the session creates the owner file, atomically, so two workers can never own one ID.
Seeds and closes take ownership, so a retried seed, a cleanup close, or a delayed seed after a close
all reach the worker that saw the ID first.
Ownership ends once its session's last record on the owner is gone:

* an agent session's ownership ends when the agent drops the session's last record,
  which is its fence or failed seed, `session_close_retry_window_seconds` after it was written;
* a resources session's ownership lasts while the session is open,
  from a successful seed until its close or `/verify`,
  and expires `session_close_retry_window_seconds` after the last request for it otherwise.
  A `/verify` that says a close follows, as an environment server's does, leaves the session open for that close.

An agent worker also owns `/rollouts/<rollout id>` from the start of a session's seed until the session closes,
so a rollout stop, which names only the rollout, reaches the worker that runs it.
That request is forwarded to the key's owner but never takes ownership of the key,
so a stop for a rollout no worker runs is handled where it arrives and leaves no owner file.

The owner files of a worker that exited stay until the server exits, so its sessions keep getting a 410.
The whole directory goes with the server's runtime directory when the server exits.
With one worker there is no private socket and no owner.

A request without a session ID that belongs to one worker, such as an agent's call to itself within one `/run`,
names that worker in the `x-ng-session-owner` header, and is forwarded there, or gets a 410 if that worker is gone.

A partial-rollout checkpoint restore installs each restored session on one live worker,
which rewrites the session's owner file to name itself (see `SessionOwners.rewrite`).
A cookie or MCP token from before the restore names only the session ID, so its requests reach the new holder.

## Module Contents

### Classes

| Name                                                                                                               | Description                                                                                                        |
| ------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
| [`AgentSessionHolder`](#nemo_gym-server_runtime-request_handling-session_routing-AgentSessionHolder)               | An agent server, whose sessions are owned by `agent_session_id` across workers.                                    |
| [`ResourcesSessionLifetimes`](#nemo_gym-server_runtime-request_handling-session_routing-ResourcesSessionLifetimes) | How long a resources server's worker keeps ownership of a session, judged from the requests it served.             |
| [`SessionKind`](#nemo_gym-server_runtime-request_handling-session_routing-SessionKind)                             | Where one kind of server's requests name their session.                                                            |
| [`SessionLifetimes`](#nemo_gym-server_runtime-request_handling-session_routing-SessionLifetimes)                   | How long a worker keeps ownership of the sessions it created.                                                      |
| [`SessionOwners`](#nemo_gym-server_runtime-request_handling-session_routing-SessionOwners)                         | Which worker currently owns each session, shared by the workers of one server.                                     |
| [`SessionRoutingMiddleware`](#nemo_gym-server_runtime-request_handling-session_routing-SessionRoutingMiddleware)   | Find a request's session ID, and forward the request to the worker that owns the session.                          |
| [`_AgentSessionLifetimes`](#nemo_gym-server_runtime-request_handling-session_routing-_AgentSessionLifetimes)       | An agent's own records decide its ownership: it releases a session when it drops the session's last record.        |
| [`_PrivateSocketProtocol`](#nemo_gym-server_runtime-request_handling-session_routing-_PrivateSocketProtocol)       | uvicorn's httptools protocol without access logs, which it would otherwise write whenever the logger has handlers. |

### Functions

| Name                                                                                                               | Description                                                                                               |
| ------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
| [`_conflict`](#nemo_gym-server_runtime-request_handling-session_routing-_conflict)                                 | -                                                                                                         |
| [`_mark_forwarded`](#nemo_gym-server_runtime-request_handling-session_routing-_mark_forwarded)                     | -                                                                                                         |
| [`_paths`](#nemo_gym-server_runtime-request_handling-session_routing-_paths)                                       | Match any of *paths*, bare or under a rollout prefix.                                                     |
| [`_read_body`](#nemo_gym-server_runtime-request_handling-session_routing-_read_body)                               | Read the request body, or None if the client disconnected.                                                |
| [`_replay_body`](#nemo_gym-server_runtime-request_handling-session_routing-_replay_body)                           | A receive that gives the app the body already read, then passes later messages, such as a disconnect, on. |
| [`_session_id`](#nemo_gym-server_runtime-request_handling-session_routing-_session_id)                             | The session ID in `field` of a seed or close body, or None for a body that has no non-empty string one.   |
| [`_sweep_ownership`](#nemo_gym-server_runtime-request_handling-session_routing-_sweep_ownership)                   | Release ownership of the sessions that have ended, so ownership ends even when no more requests arrive.   |
| [`_valid_routing_id`](#nemo_gym-server_runtime-request_handling-session_routing-_valid_routing_id)                 | -                                                                                                         |
| [`cookie_session`](#nemo_gym-server_runtime-request_handling-session_routing-cookie_session)                       | Return the session stored in a session cookie, or None if Starlette's SessionMiddleware would reject it.  |
| [`create_session_socket_dir`](#nemo_gym-server_runtime-request_handling-session_routing-create_session_socket_dir) | The directory for one server's private worker sockets and owner files: this process's runtime directory.  |
| [`install_session_routing`](#nemo_gym-server_runtime-request_handling-session_routing-install_session_routing)     | Route this worker's requests by session ID, and serve forwarded requests on a private socket.             |
| [`mcp_token_session_id`](#nemo_gym-server_runtime-request_handling-session_routing-mcp_token_session_id)           | Return the session ID in an MCP session token, or None for a token with a bad signature.                  |
| [`serve_private_socket`](#nemo_gym-server_runtime-request_handling-session_routing-serve_private_socket)           | Serve *app* on a Unix socket for requests other workers forward here, on the running event loop.          |
| [`worker_socket_path`](#nemo_gym-server_runtime-request_handling-session_routing-worker_socket_path)               | -                                                                                                         |

### Data

[`LOG`](#nemo_gym-server_runtime-request_handling-session_routing-LOG)

[`ROUTED_LIFESPANS_STATE`](#nemo_gym-server_runtime-request_handling-session_routing-ROUTED_LIFESPANS_STATE)

[`SESSION_ACQUIRE_SCOPE_KEY`](#nemo_gym-server_runtime-request_handling-session_routing-SESSION_ACQUIRE_SCOPE_KEY)

[`SESSION_ID_SCOPE_KEY`](#nemo_gym-server_runtime-request_handling-session_routing-SESSION_ID_SCOPE_KEY)

[`SESSION_OWNERS_DIR`](#nemo_gym-server_runtime-request_handling-session_routing-SESSION_OWNERS_DIR)

[`SESSION_OWNER_HEADER`](#nemo_gym-server_runtime-request_handling-session_routing-SESSION_OWNER_HEADER)

[`SESSION_SOCKET_DIR_ENV`](#nemo_gym-server_runtime-request_handling-session_routing-SESSION_SOCKET_DIR_ENV)

[`_FORWARDED_SCOPE_KEY`](#nemo_gym-server_runtime-request_handling-session_routing-_FORWARDED_SCOPE_KEY)

[`_FORWARD_KEEPALIVE_SECONDS`](#nemo_gym-server_runtime-request_handling-session_routing-_FORWARD_KEEPALIVE_SECONDS)

[`_HOP_BY_HOP_HEADERS`](#nemo_gym-server_runtime-request_handling-session_routing-_HOP_BY_HOP_HEADERS)

[`_MAX_SWEEP_SECONDS`](#nemo_gym-server_runtime-request_handling-session_routing-_MAX_SWEEP_SECONDS)

[`_MIN_SWEEP_SECONDS`](#nemo_gym-server_runtime-request_handling-session_routing-_MIN_SWEEP_SECONDS)

[`_PRIVATE_SERVER_KEEPALIVE_SECONDS`](#nemo_gym-server_runtime-request_handling-session_routing-_PRIVATE_SERVER_KEEPALIVE_SECONDS)

[`_REWRITE_BATCH`](#nemo_gym-server_runtime-request_handling-session_routing-_REWRITE_BATCH)

[`_ROLLOUT_PREFIX`](#nemo_gym-server_runtime-request_handling-session_routing-_ROLLOUT_PREFIX)

[`_ROUTING_ID_PATTERN`](#nemo_gym-server_runtime-request_handling-session_routing-_ROUTING_ID_PATTERN)

[`_SESSION_MAX_AGE_SECONDS`](#nemo_gym-server_runtime-request_handling-session_routing-_SESSION_MAX_AGE_SECONDS)

### API

```python
class nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder()
```

Protocol

An agent server, whose sessions are owned by `agent_session_id` across workers.

```python
nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder.expire_agent_session_records() -> None
```

Drop the fences and failed seeds that expired, releasing ownership of their sessions.

```python
nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder.holds_agent_session(
    agent_session_id: str
) -> bool
```

Whether this worker still keeps any record of the session, open, failed or closed.

```python
nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder.use_agent_session_owners(
    owners: nemo_gym.server_runtime.request_handling.session_routing.SessionOwners
) -> None
```

Keep the session owners, to release this worker's ownership when it drops a session's last record.

```python
class nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes(
    close_retry_window_seconds: float,
    starts: tuple[str, ...],
    ends: tuple[str, ...],
    close_follows_header: typing.Optional[str] = None
)
```

How long a resources server's worker keeps ownership of a session, judged from the requests it served.

A resources server keeps its per-session state in its own subclass, so the router judges by the requests:

* A successful seed, or another declared session start such as a Gymnasium `/reset`,
  opens the session: its ownership lasts until the session ends.
* A successful `/close_session` or `/verify`, or `SimpleResourcesServer.checkpoint_session_ended`,
  ends it, unless the request says a close follows, as an environment server's `/verify` does.
* A session that is not open keeps its owner `close_retry_window_seconds` after the last request for it.
  The caller's retried or cleanup close reaches the owner within that window.
  A delayed seed after a close reaches it too, where the server refuses it while the session is fenced.
* A failed seed or close of an open session leaves it open.

So ownership outlives the session's last request by at most the close retry window,
except for a session never closed, which keeps its owner as long as the server keeps the session.

**`_close_follows`**

---

**`_ends`** `= _paths(ends)`

---

**`_expiries`** `list[tuple[float, str]] = []`

---

**`_queued`** `set[str] = set()`

---

**`_retained`** `set[str] = set()`

---

**`_starts`** `= _paths(starts)`

---

**`_until`** `dict[str, float] = {}`

---

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.__len__() -> int
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes._close_follows_in(
    scope: starlette.types.Scope
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes._expire_after_window(
    session_id: str
) -> None
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.ended(
    session_id: str
) -> None
```

End a session this worker owns, from a route that ends a session under another name.

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.forget(
    session_id: str
) -> None
```

Drop a session whose install a failed restore undid, so its ownership can be released at once.

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.holds(
    session_id: str
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.opened(
    session_id: str
) -> None
```

Open a session a checkpoint restore installed on this worker, as a successful seed would.

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.retain(
    session_id: str
) -> None
```

Keep ownership of a session a checkpoint retire released until `ended`, when the retire's refusal ends.

Until then a late request that carries only the session's cookie reaches this worker, which refuses it.

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.served(
    session_id: str,
    scope: starlette.types.Scope,
    status: typing.Optional[int]
) -> None
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.starts(
    path: str
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ResourcesSessionLifetimes.sweep() -> list[str]
```

```python
class nemo_gym.server_runtime.request_handling.session_routing.SessionKind(
    cookie_key: str,
    id_field: str,
    seed_path: str,
    close_path: str,
    cookie_session: bool,
    path_key: typing.Optional[re.Pattern[str]] = None
)
```

Dataclass

Where one kind of server's requests name their session.

**`body_paths`** `Pattern[str]`

---

**`close_path`** `str`

---

**`cookie_key`** `str`

---

**`cookie_session`** `bool`

---

**`id_field`** `str`

---

**`path_key`** `Optional[Pattern[str]] = None`

---

**`seed_path`** `str`

---

```python
class nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes()
```

Protocol

How long a worker keeps ownership of the sessions it created.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes.holds(
    session_id: str
) -> bool
```

Whether this worker still keeps any record of the session.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes.served(
    session_id: str,
    scope: starlette.types.Scope,
    status: typing.Optional[int]
) -> None
```

Record a request for a session this worker owns, once it is served; `status` is None if it failed.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes.starts(
    path: str
) -> bool
```

Whether a request on `path` starts a session, so it takes ownership whatever its handler does.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes.sweep() -> list[str]
```

Drop the records whose time has passed, and return the sessions whose ownership can be released.

```python
class nemo_gym.server_runtime.request_handling.session_routing.SessionOwners(
    socket_dir: str,
    routing_id: str,
    holds: collections.abc.Callable[[str], bool]
)
```

Which worker currently owns each session, shared by the workers of one server.

Each session's owner is recorded in an owner file in the owners directory,
named by a hash of the session ID and holding the owner's worker ID.
It is created with a hard link from a complete temporary file, which fails if the owner file exists,
so two workers cannot both own one ID and a reader never sees a partly written owner file.

A worker owns a session as long as `holds` says it keeps anything of the session.
It does not release ownership while a request for the session is being served here,
since that request may be about to create a record.
Only the owner removes its owner file, so a removal cannot delete an owner file another worker wrote since.
A checkpoint restore can rewrite an owner file for another worker at any time, though.
The removal's check of the owner and its unlink, and every rewrite, hold one lock file shared by the workers,
so a removal never deletes an owner file a rewrite has just written.

**`_lock_path`** `= f'{self.directory}.lock'`

---

**`_mine`** `set[str] = set()`

---

**`_serving`** `dict[str, int] = {}`

---

**`directory`** `= os.path.join(socket_dir, SESSION_OWNERS_DIR)`

---

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.__len__() -> int
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners._exclusive() -> collections.abc.Iterator[None]
```

Hold the workers' owners lock; it is held only for a few file operations, never across an await.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners._path(
    session_id: str
) -> str
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners._replace_owner_files(
    session_ids: list[str],
    replaced: list[str]
) -> None
```

Replace the owner file of each ID with one naming this worker, and append each ID to `replaced` once done.

It touches only files, so it can run in a thread.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.acquire(
    session_id: str
) -> typing.Optional[str]
```

Make this worker the ID's owner unless another worker already owns it; return the owner.

Returns None for an owner file that names no valid worker, which nothing here writes.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.disown(
    session_id: str
) -> None
```

Stop owning the ID here, and leave its owner file to the worker a checkpoint restore moved it to.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.mine(
    session_id: str
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.owner(
    session_id: str
) -> typing.Optional[str]
```

The worker that owns the ID, or None if nobody does.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.release(
    session_id: str
) -> None
```

Release this worker's ownership of the ID, unless a request for it is in progress or the session remains.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.rewrite(
    session_id: str
) -> None
```

Make this worker the ID's owner whoever owns it now, for a session a checkpoint restore installs here.

The owner file is replaced by one rename, so a reader finds either the old owner or this one.
A live worker that owned the ID before must `disown` it, since a worker trusts its own ownership first.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.rewrite_all(
    session_ids: list[str]
) -> None
```

async

`rewrite` each ID, in a thread, so a restore of many sessions does not hold up the event loop.

The thread only replaces files, and holds the workers' owners lock once per `_REWRITE_BATCH` renames.
The IDs this worker owns change only on the event loop, once the thread has ended,
even if the caller was cancelled meanwhile.
They then name exactly the owner files the thread rewrote, so a release after a failure misses none of them.
Until then this worker does not count them as its own, so a release here leaves them alone.
The caller holds the sessions (see `holds`) from before this call, so no release afterwards removes them.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionOwners.serving(
    session_id: str
) -> collections.abc.Iterator[None]
```

Keep ownership while a request for the ID is served here, and release it after if nothing is left.

```python
class nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware(
    app: starlette.types.ASGIApp,
    kind: nemo_gym.server_runtime.request_handling.session_routing.SessionKind,
    session_cookie: str,
    secret_key: str,
    routing_id: typing.Optional[str] = None,
    socket_dir: typing.Optional[str] = None,
    clients: typing.Optional[dict[str, aiohttp.ClientSession]] = None,
    owners: typing.Optional[nemo_gym.server_runtime.request_handling.session_routing.SessionOwners] = None,
    lifetimes: typing.Optional[nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes] = None,
    mcp_token_header: typing.Optional[str] = None,
    mcp_token_serializer: typing.Optional[itsdangerous.URLSafeSerializer] = None,
    mcp_path: str = '/mcp'
)
```

Find a request's session ID, and forward the request to the worker that owns the session.

It sits outside SessionMiddleware and reads the cookie by decoding it with the same secret.
A forwarded request therefore never reaches this worker's SessionMiddleware,
so the reply carries only the owner's Set-Cookie, built from the session as the owner left it.
Sitting inside SessionMiddleware instead would add this worker's stale copy of the session as a second Set-Cookie.

With one worker there is no ownership, and it only reads the session ID of seeds and closes:
it refuses a request whose cookie and body name different sessions,
and gives an episode's resources session its ID in the cookie.

**`body_paths`** `= kind.body_paths`

---

**`clients`** `= clients if clients is not None else {}`

---

**`signer`** `= itsdangerous.TimestampSigner(secret_key)`

---

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware.__call__(
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send
) -> None
```

async

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware._client(
    owner: str
) -> aiohttp.ClientSession
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware._discard_client(
    owner: str
) -> None
```

async

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware._forward(
    owner: str,
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send,
    body: typing.Optional[bytes] = None
) -> None
```

async

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware._serve_owned(
    owners: nemo_gym.server_runtime.request_handling.session_routing.SessionOwners,
    session_id: str,
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send
) -> None
```

async

Serve a request for a session this worker owns, or may take, and record it for the ownership's lifetime.

```python
nemo_gym.server_runtime.request_handling.session_routing.SessionRoutingMiddleware._session_id_from_headers(
    connection: starlette.requests.HTTPConnection
) -> typing.Optional[str]
```

```python
class nemo_gym.server_runtime.request_handling.session_routing._AgentSessionLifetimes(
    agent: nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder
)
```

An agent's own records decide its ownership: it releases a session when it drops the session's last record.

```python
nemo_gym.server_runtime.request_handling.session_routing._AgentSessionLifetimes.holds(
    session_id: str
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing._AgentSessionLifetimes.served(
    session_id: str,
    scope: starlette.types.Scope,
    status: typing.Optional[int]
) -> None
```

```python
nemo_gym.server_runtime.request_handling.session_routing._AgentSessionLifetimes.starts(
    path: str
) -> bool
```

```python
nemo_gym.server_runtime.request_handling.session_routing._AgentSessionLifetimes.sweep() -> list[str]
```

```python
class nemo_gym.server_runtime.request_handling.session_routing._PrivateSocketProtocol(
    args: typing.Any = (),
    kwargs: typing.Any = {}
)
```

Protocol

**Bases:** `HttpToolsProtocol`

uvicorn's httptools protocol without access logs, which it would otherwise write whenever the logger has handlers.

```python
nemo_gym.server_runtime.request_handling.session_routing._conflict(
    detail: str,
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send
) -> None
```

async

```python
nemo_gym.server_runtime.request_handling.session_routing._mark_forwarded(
    app: starlette.types.ASGIApp
) -> starlette.types.ASGIApp
```

```python
nemo_gym.server_runtime.request_handling.session_routing._paths(
    paths: tuple[str, ...]
) -> re.Pattern[str]
```

Match any of *paths*, bare or under a rollout prefix.

```python
nemo_gym.server_runtime.request_handling.session_routing._read_body(
    receive: starlette.types.Receive
) -> typing.Optional[bytes]
```

async

Read the request body, or None if the client disconnected.

```python
nemo_gym.server_runtime.request_handling.session_routing._replay_body(
    body: bytes,
    receive: starlette.types.Receive
) -> starlette.types.Receive
```

A receive that gives the app the body already read, then passes later messages, such as a disconnect, on.

```python
nemo_gym.server_runtime.request_handling.session_routing._session_id(
    body: bytes,
    field: str
) -> typing.Optional[str]
```

The session ID in `field` of a seed or close body, or None for a body that has no non-empty string one.

```python
nemo_gym.server_runtime.request_handling.session_routing._sweep_ownership(
    owners: nemo_gym.server_runtime.request_handling.session_routing.SessionOwners,
    lifetimes: nemo_gym.server_runtime.request_handling.session_routing.SessionLifetimes,
    interval_seconds: float
) -> None
```

async

Release ownership of the sessions that have ended, so ownership ends even when no more requests arrive.

```python
nemo_gym.server_runtime.request_handling.session_routing._valid_routing_id(
    owner: typing.Any
) -> typing.Optional[str]
```

```python
nemo_gym.server_runtime.request_handling.session_routing.cookie_session(
    cookie: str,
    signer: itsdangerous.TimestampSigner
) -> typing.Optional[dict[str, typing.Any]]
```

Return the session stored in a session cookie, or None if Starlette's SessionMiddleware would reject it.

Decoded exactly as SessionMiddleware decodes it.
A cookie it rejects (badly signed or expired) starts a fresh session.

```python
nemo_gym.server_runtime.request_handling.session_routing.create_session_socket_dir() -> str
```

The directory for one server's private worker sockets and owner files: this process's runtime directory.

```python
nemo_gym.server_runtime.request_handling.session_routing.install_session_routing(
    app: fastapi.FastAPI,
    kind: nemo_gym.server_runtime.request_handling.session_routing.SessionKind,
    socket_dir: typing.Optional[str],
    session_cookie: str,
    secret_key: str,
    mcp_token_header: typing.Optional[str] = None,
    mcp_token_serializer: typing.Optional[itsdangerous.URLSafeSerializer] = None,
    agent_sessions: typing.Optional[nemo_gym.server_runtime.request_handling.session_routing.AgentSessionHolder] = None,
    close_retry_window_seconds: typing.Optional[float] = None,
    session_starts: tuple[str, ...] = (),
    session_ends: tuple[str, ...] = (),
    close_follows_header: typing.Optional[str] = None,
    backlog: int = DEFAULT_UVICORN_BACKLOG
) -> typing.Optional[str]
```

Route this worker's requests by session ID, and serve forwarded requests on a private socket.

Call once the app's middleware is otherwise in place: the router must sit outside SessionMiddleware.
Without a `socket_dir` (one worker), it only reads the session ID of seeds and closes, and owns nothing.
With one, it records session owners in it and returns this worker's ID,
also stored in `app.state.nemo_gym_routing_id`.
Pass the agent for an agent server, whose own records decide how long it owns its sessions,
or the close retry window and the session start and end routes for a resources server
(see `ResourcesSessionLifetimes`).

```python
nemo_gym.server_runtime.request_handling.session_routing.mcp_token_session_id(
    token: str,
    serializer: itsdangerous.URLSafeSerializer
) -> typing.Optional[str]
```

Return the session ID in an MCP session token, or None for a token with a bad signature.

A rejected token is handled locally, where the MCP endpoint's own token check refuses it.

```python
nemo_gym.server_runtime.request_handling.session_routing.serve_private_socket(
    app: starlette.types.ASGIApp,
    path: str,
    backlog: int = DEFAULT_UVICORN_BACKLOG
) -> collections.abc.AsyncIterator[None]
```

async

Serve *app* on a Unix socket for requests other workers forward here, on the running event loop.

```python
nemo_gym.server_runtime.request_handling.session_routing.worker_socket_path(
    socket_dir: str,
    routing_id: str
) -> str
```

```python
nemo_gym.server_runtime.request_handling.session_routing.LOG = logging.getLogger(__name__)
```

```python
nemo_gym.server_runtime.request_handling.session_routing.ROUTED_LIFESPANS_STATE = 'nemo_gym_routed_lifespans'
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SESSION_ACQUIRE_SCOPE_KEY = 'nemo_gym.acquire_session'
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SESSION_ID_SCOPE_KEY = 'nemo_gym.session_id'
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SESSION_OWNERS_DIR = 'session-owners'
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SESSION_OWNER_HEADER = 'x-ng-session-owner'
```

```python
nemo_gym.server_runtime.request_handling.session_routing.SESSION_SOCKET_DIR_ENV = 'NEMO_GYM_SESSION_SOCKET_DIR'
```

```python
nemo_gym.server_runtime.request_handling.session_routing._FORWARDED_SCOPE_KEY = 'nemo_gym.session_forwarded'
```

```python
nemo_gym.server_runtime.request_handling.session_routing._FORWARD_KEEPALIVE_SECONDS = 15.0
```

```python
nemo_gym.server_runtime.request_handling.session_routing._HOP_BY_HOP_HEADERS = frozenset({'connection', 'keep-alive', 'transfer-encoding', 'content-length', 'u...
```

```python
nemo_gym.server_runtime.request_handling.session_routing._MAX_SWEEP_SECONDS = 30.0
```

```python
nemo_gym.server_runtime.request_handling.session_routing._MIN_SWEEP_SECONDS = 0.05
```

```python
nemo_gym.server_runtime.request_handling.session_routing._PRIVATE_SERVER_KEEPALIVE_SECONDS = 30
```

```python
nemo_gym.server_runtime.request_handling.session_routing._REWRITE_BATCH = 256
```

```python
nemo_gym.server_runtime.request_handling.session_routing._ROLLOUT_PREFIX = f'(?:/{ROLLOUT_PATH_PREFIX}/[^/]+(?:/[^/]+)?)?'
```

```python
nemo_gym.server_runtime.request_handling.session_routing._ROUTING_ID_PATTERN = re.compile('[0-9a-f]{32}')
```

```python
nemo_gym.server_runtime.request_handling.session_routing._SESSION_MAX_AGE_SECONDS = 14 * 24 * 60 * 60
```