> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.server_utils

## Module Contents

### Classes

| Name                                                                                                      | Description                                                                   |
| --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [`BaseServer`](#nemo_gym-server_utils-BaseServer)                                                         | All instances of BaseServer are queryable using ServerClient.                 |
| [`ClientDisconnectCancellationMiddleware`](#nemo_gym-server_utils-ClientDisconnectCancellationMiddleware) | Cancel an in-flight HTTP request when its client disconnects.                 |
| [`GlobalAIOHTTPAsyncClientConfig`](#nemo_gym-server_utils-GlobalAIOHTTPAsyncClientConfig)                 | -                                                                             |
| [`HeadServer`](#nemo_gym-server_utils-HeadServer)                                                         | -                                                                             |
| [`KeepaliveHttpToolsProtocol`](#nemo_gym-server_utils-KeepaliveHttpToolsProtocol)                         | Uvicorn's httptools protocol with TCP keepalive on every accepted connection. |
| [`ProfilingMiddlewareConfig`](#nemo_gym-server_utils-ProfilingMiddlewareConfig)                           | -                                                                             |
| [`ProfilingMiddlewareInputConfig`](#nemo_gym-server_utils-ProfilingMiddlewareInputConfig)                 | -                                                                             |
| [`ServerClient`](#nemo_gym-server_utils-ServerClient)                                                     | -                                                                             |
| [`ServerInstanceDisplayConfig`](#nemo_gym-server_utils-ServerInstanceDisplayConfig)                       | -                                                                             |
| [`SimpleServer`](#nemo_gym-server_utils-SimpleServer)                                                     | -                                                                             |
| [`UvicornLoggingConfig`](#nemo_gym-server_utils-UvicornLoggingConfig)                                     | -                                                                             |
| [`UvicornProxyHeadersConfig`](#nemo_gym-server_utils-UvicornProxyHeadersConfig)                           | -                                                                             |
| [`_PickleSafeRequestInfo`](#nemo_gym-server_utils-_PickleSafeRequestInfo)                                 | -                                                                             |

### Functions

| Name                                                                                                                      | Description                                                                    |
| ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| [`_bounded_validation_error_value`](#nemo_gym-server_utils-_bounded_validation_error_value)                               | -                                                                              |
| [`_escaped_log_prefix`](#nemo_gym-server_utils-_escaped_log_prefix)                                                       | Return a quoted, control-safe prefix bounded by `max_chars`.                   |
| [`_escaped_log_text`](#nemo_gym-server_utils-_escaped_log_text)                                                           | Return bounded, control-safe text with a visible marker when truncated.        |
| [`_format_upstream_error_body`](#nemo_gym-server_utils-_format_upstream_error_body)                                       | Render a bounded upstream error body without hiding its useful traceback.      |
| [`_format_upstream_error_log`](#nemo_gym-server_utils-_format_upstream_error_log)                                         | -                                                                              |
| [`_get_ray`](#nemo_gym-server_utils-_get_ray)                                                                             | Import Ray only for processes configured to use it.                            |
| [`_has_injected_global_config_env`](#nemo_gym-server_utils-_has_injected_global_config_env)                               | -                                                                              |
| [`_log_validation_exception`](#nemo_gym-server_utils-_log_validation_exception)                                           | -                                                                              |
| [`_make_keepalive_socket_factory`](#nemo_gym-server_utils-_make_keepalive_socket_factory)                                 | -                                                                              |
| [`_redacted_url`](#nemo_gym-server_utils-_redacted_url)                                                                   | Strip the query string from a URL before it becomes a span attribute.          |
| [`_request_with_retries`](#nemo_gym-server_utils-_request_with_retries)                                                   | -                                                                              |
| [`_server_uses_ray`](#nemo_gym-server_utils-_server_uses_ray)                                                             | -                                                                              |
| [`_set_tcp_keepalive`](#nemo_gym-server_utils-_set_tcp_keepalive)                                                         | -                                                                              |
| [`_telemetry_server_type`](#nemo_gym-server_utils-_telemetry_server_type)                                                 | Return the Gym server-type name for *server\_cls*, or None if it is not one.   |
| [`_traced_request`](#nemo_gym-server_utils-_traced_request)                                                               | `_request_with_retries` wrapped in a CLIENT span, with `traceparent` injected. |
| [`_validation_error_summaries`](#nemo_gym-server_utils-_validation_error_summaries)                                       | -                                                                              |
| [`_validation_exception_handler`](#nemo_gym-server_utils-_validation_exception_handler)                                   | -                                                                              |
| [`apply_rollout_prefix`](#nemo_gym-server_utils-apply_rollout_prefix)                                                     | Append a rollout prefix to a model-server root URL.                            |
| [`get_global_aiohttp_client`](#nemo_gym-server_utils-get_global_aiohttp_client)                                           | -                                                                              |
| [`get_nemo_gym_fastapi_num_workers`](#nemo_gym-server_utils-get_nemo_gym_fastapi_num_workers)                             | -                                                                              |
| [`get_response_json`](#nemo_gym-server_utils-get_response_json)                                                           | -                                                                              |
| [`get_server_url`](#nemo_gym-server_utils-get_server_url)                                                                 | -                                                                              |
| [`global_aiohttp_client_exit`](#nemo_gym-server_utils-global_aiohttp_client_exit)                                         | -                                                                              |
| [`initialize_ray`](#nemo_gym-server_utils-initialize_ray)                                                                 | Initialize ray cluster in a process.                                           |
| [`is_global_aiohttp_client_request_debug_enabled`](#nemo_gym-server_utils-is_global_aiohttp_client_request_debug_enabled) | -                                                                              |
| [`is_global_aiohttp_client_setup`](#nemo_gym-server_utils-is_global_aiohttp_client_setup)                                 | -                                                                              |
| [`is_nemo_gym_fastapi_entrypoint`](#nemo_gym-server_utils-is_nemo_gym_fastapi_entrypoint)                                 | -                                                                              |
| [`is_nemo_gym_fastapi_worker`](#nemo_gym-server_utils-is_nemo_gym_fastapi_worker)                                         | -                                                                              |
| [`maybe_ray_cluster_exit`](#nemo_gym-server_utils-maybe_ray_cluster_exit)                                                 | -                                                                              |
| [`raise_for_status`](#nemo_gym-server_utils-raise_for_status)                                                             | -                                                                              |
| [`request`](#nemo_gym-server_utils-request)                                                                               | Make an outbound HTTP call through Gym's shared aiohttp client.                |
| [`rollout_path_prefix`](#nemo_gym-server_utils-rollout_path_prefix)                                                       | Return the leading model-server path prefix for a rollout, if available.       |
| [`set_global_aiohttp_client`](#nemo_gym-server_utils-set_global_aiohttp_client)                                           | -                                                                              |
| [`set_is_nemo_gym_fastapi_entrypoint`](#nemo_gym-server_utils-set_is_nemo_gym_fastapi_entrypoint)                         | -                                                                              |
| [`set_is_nemo_gym_fastapi_worker`](#nemo_gym-server_utils-set_is_nemo_gym_fastapi_worker)                                 | -                                                                              |
| [`set_nemo_gym_fastapi_num_workers`](#nemo_gym-server_utils-set_nemo_gym_fastapi_num_workers)                             | -                                                                              |
| [`setup_server_client`](#nemo_gym-server_utils-setup_server_client)                                                       | -                                                                              |

### Data

[`DEFAULT_HEAD_SERVER_PORT`](#nemo_gym-server_utils-DEFAULT_HEAD_SERVER_PORT)

[`DISCONNECTED_CLIENT_OS_HELP_TEXT`](#nemo_gym-server_utils-DISCONNECTED_CLIENT_OS_HELP_TEXT)

[`DISCONNECTED_CLIENT_OS_PRINT_INTERVAL`](#nemo_gym-server_utils-DISCONNECTED_CLIENT_OS_PRINT_INTERVAL)

[`IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME`](#nemo_gym-server_utils-IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME)

[`IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME`](#nemo_gym-server_utils-IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME)

[`MAX_NUM_TRIES`](#nemo_gym-server_utils-MAX_NUM_TRIES)

[`NEMO_GYM_FASTAPI_NUM_WORKERS`](#nemo_gym-server_utils-NEMO_GYM_FASTAPI_NUM_WORKERS)

[`NEMO_GYM_MODEL_SERVER_BASE_URL_ENV_VAR_NAME`](#nemo_gym-server_utils-NEMO_GYM_MODEL_SERVER_BASE_URL_ENV_VAR_NAME)

[`NEMO_GYM_MODEL_SERVER_NAME_ENV_VAR_NAME`](#nemo_gym-server_utils-NEMO_GYM_MODEL_SERVER_NAME_ENV_VAR_NAME)

[`SESSION_ID_KEY`](#nemo_gym-server_utils-SESSION_ID_KEY)

[`ServerStatus`](#nemo_gym-server_utils-ServerStatus)

[`_ALL_V4_MAPPED`](#nemo_gym-server_utils-_ALL_V4_MAPPED)

[`_GLOBAL_AIOHTTP_CLIENT`](#nemo_gym-server_utils-_GLOBAL_AIOHTTP_CLIENT)

[`_GLOBAL_AIOHTTP_CLIENT_QUEUE_TELEMETRY`](#nemo_gym-server_utils-_GLOBAL_AIOHTTP_CLIENT_QUEUE_TELEMETRY)

[`_GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG`](#nemo_gym-server_utils-_GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG)

[`_NEMO_GYM_STARTED_RAY_CLUSTER`](#nemo_gym-server_utils-_NEMO_GYM_STARTED_RAY_CLUSTER)

[`_NUM_CLIENT_OS_ERROR`](#nemo_gym-server_utils-_NUM_CLIENT_OS_ERROR)

[`_NUM_SERVER_DISCONNECTED_ERROR`](#nemo_gym-server_utils-_NUM_SERVER_DISCONNECTED_ERROR)

[`_TELEMETRY_SERVER_TYPE_BY_BASE`](#nemo_gym-server_utils-_TELEMETRY_SERVER_TYPE_BY_BASE)

[`_UPSTREAM_ERROR_LOG_BODY_CHARS`](#nemo_gym-server_utils-_UPSTREAM_ERROR_LOG_BODY_CHARS)

[`_VALIDATION_ERROR_LOG_BODY_CHARS`](#nemo_gym-server_utils-_VALIDATION_ERROR_LOG_BODY_CHARS)

[`_VALIDATION_ERROR_LOG_FIELD_CHARS`](#nemo_gym-server_utils-_VALIDATION_ERROR_LOG_FIELD_CHARS)

[`_VALIDATION_ERROR_LOG_LOC_ITEMS`](#nemo_gym-server_utils-_VALIDATION_ERROR_LOG_LOC_ITEMS)

[`_VALIDATION_ERROR_LOG_MAX_ERRORS`](#nemo_gym-server_utils-_VALIDATION_ERROR_LOG_MAX_ERRORS)

[`_WARNED_IMPLICIT_RAY_SERVERS`](#nemo_gym-server_utils-_WARNED_IMPLICIT_RAY_SERVERS)

[`logger`](#nemo_gym-server_utils-logger)

### API

```python
class nemo_gym.server_utils.BaseServer()
```

**Bases:** `BaseModel`

All instances of BaseServer are queryable using ServerClient.

**`config`** `BaseRunServerInstanceConfig`

---

```python
nemo_gym.server_utils.BaseServer.load_config_from_global_config() -> nemo_gym.config_types.BaseRunServerInstanceConfig
```

classmethod

```python
nemo_gym.server_utils.BaseServer.setup_liveness(
    app: fastapi.FastAPI
) -> None
```

```python
class nemo_gym.server_utils.ClientDisconnectCancellationMiddleware(
    app: starlette.types.ASGIApp
)
```

Cancel an in-flight HTTP request when its client disconnects.

**`num_cancelled`** `= 0`

---

```python
nemo_gym.server_utils.ClientDisconnectCancellationMiddleware.__call__(
    scope: starlette.types.Scope,
    receive: starlette.types.Receive,
    send: starlette.types.Send
) -> None
```

async

```python
class nemo_gym.server_utils.GlobalAIOHTTPAsyncClientConfig()
```

**Bases:** `BaseModel`

**`global_aiohttp_client_request_debug`** `bool = False`

---

**`global_aiohttp_connector_limit`** `int`

---

**`global_aiohttp_connector_limit_per_host`** `int`

---

**`global_aiohttp_intended_concurrency`** `Optional[int]`

---

**`global_aiohttp_intended_concurrency_per_host`** `Optional[int]`

---

**`global_aiohttp_tcp_keepalive_idle_seconds`** `int`

---

**`global_aiohttp_tcp_keepalive_interval_seconds`** `int`

---

**`global_aiohttp_tcp_keepalive_probes`** `int`

---

```python
class nemo_gym.server_utils.HeadServer()
```

**Bases:** [BaseServer](#nemo_gym-server_utils-BaseServer)

**`_cached_yaml`** `Optional[str] = None`

---

**`_ready`** `bool = PrivateAttr(default=False)`

---

**`_server_instances`** `List[dict] = []`

---

**`config`** `BaseServerConfig`

---

```python
nemo_gym.server_utils.HeadServer.get_server_instances() -> typing.List[dict]
```

```python
nemo_gym.server_utils.HeadServer.global_config_dict_yaml() -> str
```

async

```python
nemo_gym.server_utils.HeadServer.invalidate_global_config_dict_yaml_cache() -> None
```

Clear the serialized global config cache.

```python
nemo_gym.server_utils.HeadServer.mark_ready() -> None
```

```python
nemo_gym.server_utils.HeadServer.run_webserver() -> typing.Tuple[uvicorn.Server, threading.Thread, nemo_gym.server_utils.HeadServer]
```

classmethod

```python
nemo_gym.server_utils.HeadServer.set_server_instances(
    instances: typing.List
) -> None
```

```python
nemo_gym.server_utils.HeadServer.setup_webserver() -> fastapi.FastAPI
```

```python
class nemo_gym.server_utils.KeepaliveHttpToolsProtocol(
    args: typing.Any = (),
    keepalive: typing.Tuple[int, int, int],
    kwargs: typing.Any = {}
)
```

Protocol

**Bases:** `HttpToolsProtocol`

Uvicorn's httptools protocol with TCP keepalive on every accepted connection.

A model server answers only after generation finishes, so a client connection can carry no bytes for many
minutes. Stateful network hops on some paths evict flows that stay idle that long, which silently drops the
eventual reply and leaves the client waiting on a half-open connection. Keepalive probes keep the flow alive and
let the kernel reap peers that are really gone. Uses the same `global_aiohttp_tcp_keepalive_*` settings as the
outgoing client connections.

Handed to uvicorn as `partial(KeepaliveHttpToolsProtocol, keepalive=(idle, interval, probes))`: multi-worker
uvicorn pickles its config into worker processes, so this class must stay importable at module level.

```python
nemo_gym.server_utils.KeepaliveHttpToolsProtocol.connection_made(
    transport: asyncio.Transport
) -> None
```

```python
class nemo_gym.server_utils.ProfilingMiddlewareConfig()
```

**Bases:** [ProfilingMiddlewareInputConfig](#nemo_gym-server_utils-ProfilingMiddlewareInputConfig)

**`profiling_enabled`** `bool = False`

---

```python
class nemo_gym.server_utils.ProfilingMiddlewareInputConfig()
```

**Bases:** `BaseModel`

**`profiling_results_dirpath`** `Optional[str] = None`

---

```python
class nemo_gym.server_utils.ServerClient()
```

**Bases:** `BaseModel`

**`_server_base_urls`** `dict[str, str] = PrivateAttr(default_factory=dict)`

---

**`global_config_dict`** `DictConfig`

---

**`head_server_config`** `BaseServerConfig`

---

**`model_config`** `= ConfigDict(arbitrary_types_allowed=True)`

---

```python
nemo_gym.server_utils.ServerClient._build_server_base_url(
    server_config_dict: omegaconf.OmegaConf
) -> str
```

```python
nemo_gym.server_utils.ServerClient._resolve_base_url(
    server_name: str
) -> str
```

```python
nemo_gym.server_utils.ServerClient.get(
    server_name: str,
    url_path: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

Args:
server\_name: str
The name of the server you are trying to call.
url\_path: str
The URL path in the server you are trying to call e.g. "/v1/responses".

```python
nemo_gym.server_utils.ServerClient.load_from_global_config(
    head_server_config: typing.Optional[nemo_gym.config_types.BaseServerConfig] = None
) -> nemo_gym.server_utils.ServerClient
```

classmethod

Build a client from the fully resolved global config.

Gym-launched server processes reuse the config injected by their parent.
Other processes fetch the config from the head server.

```python
nemo_gym.server_utils.ServerClient.load_head_server_config() -> nemo_gym.config_types.BaseServerConfig
```

classmethod

```python
nemo_gym.server_utils.ServerClient.poll_for_status(
    server_name: str,
    timeout_seconds: float = 5
) -> nemo_gym.server_utils.ServerStatus
```

```python
nemo_gym.server_utils.ServerClient.post(
    server_name: str,
    url_path: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

Args:
server\_name: str
The name of the server you are trying to call.
url\_path: str
The URL path in the server you are trying to call e.g. "/v1/responses".

```python
nemo_gym.server_utils.ServerClient.request(
    server_name: str,
    url_path: str,
    method: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

```python
class nemo_gym.server_utils.ServerInstanceDisplayConfig()
```

**Bases:** `BaseModel`

**`config_path`** `Optional[str] = None`

---

**`dir_path`** `Optional[Path] = None`

---

**`entrypoint`** `Optional[str] = None`

---

**`host`** `Optional[str] = None`

---

**`name`** `Optional[str] = None`

---

**`pid`** `Optional[int] = None`

---

**`port`** `Optional[int] = None`

---

**`process_name`** `Optional[str] = None`

---

**`server_type`** `Optional[str] = None`

---

**`start_time`** `Optional[float] = None`

---

**`status`** `Optional[ServerStatus] = None`

---

**`uptime_seconds`** `Optional[float] = None`

---

**`url`** `Optional[str] = None`

---

```python
class nemo_gym.server_utils.SimpleServer()
```

**Bases:** [BaseServer](#nemo_gym-server_utils-BaseServer)

**`ray_enabled`** `bool | None = None`

---

**`server_client`** `ServerClient`

---

```python
nemo_gym.server_utils.SimpleServer.get_session_middleware_key() -> str
```

```python
nemo_gym.server_utils.SimpleServer.instrument_app_for_telemetry(
    app: fastapi.FastAPI
) -> None
```

Apply OTel FastAPI auto-instrumentation to *app*, if telemetry is exporting.

Gives every server the inbound half of context propagation plus dimensioned
`http.server.*` metrics, which is why Gym does not use nemo-lens's undimensioned
`gym.server.request_duration_ms` (see `nemo_gym/telemetry/metrics.py`).

A server whose instrumentation could not be applied keeps serving; it just does not
extract inbound trace context.

```python
nemo_gym.server_utils.SimpleServer.prefix_server_logs() -> None
```

```python
nemo_gym.server_utils.SimpleServer.run_webserver() -> typing.Optional[fastapi.FastAPI]
```

classmethod

```python
nemo_gym.server_utils.SimpleServer.set_ulimit(
    target_soft_limit: int = 65535
)
```

```python
nemo_gym.server_utils.SimpleServer.setup_cancellation_middleware(
    app: fastapi.FastAPI
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_exception_middleware(
    app: fastapi.FastAPI
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_profiling(
    app: fastapi.FastAPI,
    profiling_config: nemo_gym.server_utils.ProfilingMiddlewareConfig
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_session_middleware(
    app: fastapi.FastAPI
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_telemetry() -> None
```

Initialise this process's nemo-lens telemetry. Idempotent, once per process.

Every Gym server is its own process with its own providers — there is no parent
handle to inherit, only the `NEMO_GYM_OTEL_*` environment the orchestrator
exported before spawning it. A failure here is logged and swallowed: telemetry
must never stop a server from serving.

```python
nemo_gym.server_utils.SimpleServer.setup_webserver() -> fastapi.FastAPI
```

abstract

```python
class nemo_gym.server_utils.UvicornLoggingConfig()
```

**Bases:** `BaseModel`

**`uvicorn_logging_show_200_ok`** `bool = False`

---

```python
class nemo_gym.server_utils.UvicornProxyHeadersConfig()
```

**Bases:** `BaseModel`

**`uvicorn_forwarded_allow_ips`** `Optional[List[str]] = None`

---

**`uvicorn_proxy_headers`** `bool = False`

---

```python
nemo_gym.server_utils.UvicornProxyHeadersConfig._require_trusted_proxy_allowlist() -> nemo_gym.server_utils.UvicornProxyHeadersConfig
```

```python
class nemo_gym.server_utils._PickleSafeRequestInfo()
```

**Bases:** `NamedTuple`

**`headers`** `CIMultiDict[str]`

---

**`method`** `str`

---

**`real_url`** `str`

---

**`url`** `str`

---

```python
nemo_gym.server_utils._bounded_validation_error_value(
    value: typing.Any
) -> tuple[typing.Any, bool]
```

```python
nemo_gym.server_utils._escaped_log_prefix(
    value: str,
    max_chars: int
) -> tuple[str, bool]
```

Return a quoted, control-safe prefix bounded by `max_chars`.

```python
nemo_gym.server_utils._escaped_log_text(
    value: str,
    max_chars: int
) -> tuple[str, bool]
```

Return bounded, control-safe text with a visible marker when truncated.

```python
nemo_gym.server_utils._format_upstream_error_body(
    content: typing.Any
) -> str
```

Render a bounded upstream error body without hiding its useful traceback.

```python
nemo_gym.server_utils._format_upstream_error_log(
    server_name: str,
    error: aiohttp.ClientResponseError
) -> str
```

```python
nemo_gym.server_utils._get_ray()
```

Import Ray only for processes configured to use it.

```python
nemo_gym.server_utils._has_injected_global_config_env() -> bool
```

```python
nemo_gym.server_utils._log_validation_exception(
    request: fastapi.Request,
    exc: fastapi.exceptions.RequestValidationError
) -> None
```

async

```python
nemo_gym.server_utils._make_keepalive_socket_factory(
    idle_seconds: int,
    interval_seconds: int,
    probes: int
)
```

```python
nemo_gym.server_utils._redacted_url(
    url: str
) -> str
```

Strip the query string from a URL before it becomes a span attribute.

Query strings in Gym carry API keys and, on rollout-prefixed routes, task content.
The path is the useful part for a trace; the query is a leak waiting to happen.

```python
nemo_gym.server_utils._request_with_retries(
    method: str,
    url: str,
    _internal: bool = False,
    _max_connection_retries: typing.Optional[int] = None,
    _server_name: typing.Optional[str] = None,
    _max_num_tries: typing.Optional[int] = None,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

```python
nemo_gym.server_utils._server_uses_ray(
    server_class: type
) -> bool
```

```python
nemo_gym.server_utils._set_tcp_keepalive(
    sock: socket.socket,
    idle_seconds: int,
    interval_seconds: int,
    probes: int
) -> None
```

```python
nemo_gym.server_utils._telemetry_server_type(
    server_cls: typing.Type
) -> typing.Optional[str]
```

Return the Gym server-type name for *server\_cls*, or None if it is not one.

```python
nemo_gym.server_utils._traced_request(
    method: str,
    url: str,
    _internal: bool = False,
    _max_connection_retries: typing.Optional[int] = None,
    _server_name: typing.Optional[str] = None,
    _max_num_tries: typing.Optional[int] = None,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

`_request_with_retries` wrapped in a CLIENT span, with `traceparent` injected.

Uses `nemo_gym.telemetry.spans.client_span` rather than nemo-lens's `managed_span`
because the latter cannot set `SpanKind` — see that module for why an INTERNAL span
is wrong on a cross-service hop.

Injection happens inside the span so the header carries *this* span as the parent —
the receiving server's FastAPI instrumentation extracts it and its SERVER span becomes
our child. That edge is what makes one rollout a single trace across three processes.

Retries reuse the same span rather than starting one per attempt: the caller asked for
one logical request, and the retry count is recorded as an attribute instead.

```python
nemo_gym.server_utils._validation_error_summaries(
    errors: list[dict[str, typing.Any]]
) -> tuple[list[dict[str, typing.Any]], bool]
```

```python
nemo_gym.server_utils._validation_exception_handler(
    request: fastapi.Request,
    exc: fastapi.exceptions.RequestValidationError
) -> fastapi.Response
```

async

```python
nemo_gym.server_utils.apply_rollout_prefix(
    base_url: str,
    rollout_id: typing.Optional[str],
    token_capture: bool = False
) -> str
```

Append a rollout prefix to a model-server root URL.

```python
nemo_gym.server_utils.get_global_aiohttp_client(
    global_config_dict_parser_config: typing.Optional[nemo_gym.global_config.GlobalConfigDictParserConfig] = None,
    global_config_dict_parser_cls: typing.Type[nemo_gym.global_config.GlobalConfigDictParser] = GlobalConfigDictParser
) -> aiohttp.ClientSession
```

```python
nemo_gym.server_utils.get_nemo_gym_fastapi_num_workers() -> int
```

```python
nemo_gym.server_utils.get_response_json(
    response: aiohttp.ClientResponse
) -> typing.Any
```

async

```python
nemo_gym.server_utils.get_server_url(
    server_name: str
) -> str
```

```python
nemo_gym.server_utils.global_aiohttp_client_exit()
```

```python
nemo_gym.server_utils.initialize_ray() -> None
```

Initialize ray cluster in a process.
We store the Ray address in the global config dict so that child processes can connect to it.
This avoids the need to start a new Ray cluster in each child process.
Note: This function will modify the global config dict - update `ray_head_node_address`

```python
nemo_gym.server_utils.is_global_aiohttp_client_request_debug_enabled() -> bool
```

```python
nemo_gym.server_utils.is_global_aiohttp_client_setup() -> bool
```

```python
nemo_gym.server_utils.is_nemo_gym_fastapi_entrypoint(
    file: str
) -> bool
```

```python
nemo_gym.server_utils.is_nemo_gym_fastapi_worker() -> bool
```

```python
nemo_gym.server_utils.maybe_ray_cluster_exit()
```

```python
nemo_gym.server_utils.raise_for_status(
    response: aiohttp.ClientResponse,
    content: typing.Optional[bytes] = None
) -> None
```

async

```python
nemo_gym.server_utils.request(
    method: str,
    url: str,
    _internal: bool = False,
    _max_connection_retries: typing.Optional[int] = None,
    _server_name: typing.Optional[str] = None,
    _max_num_tries: typing.Optional[int] = None,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

Make an outbound HTTP call through Gym's shared aiohttp client.

This is the only place Gym talks to another server, so it is also the only place
trace context has to be injected: every agent -> model and agent -> resources hop goes
through here. `CLAUDE.md` bans httpx precisely to keep it that way.

`_server_name` is the bounded logical destination label for pool metrics: a configured
`ServerClient` server name, `remote_agent_service` for the remote agent's external
service, or `None` for the fallback label `external`. It is retained across retries
and redirects and is not forwarded to aiohttp.

`_max_num_tries` caps this call's total attempts on every exception path. It replaces the
default generic-error limit (`MAX_NUM_TRIES` attempts for external calls, unbounded for
internal ones). A `_max_connection_retries` limit still applies as well.

```python
nemo_gym.server_utils.rollout_path_prefix(
    rollout_id: typing.Optional[str],
    token_capture: bool = False
) -> str
```

Return the leading model-server path prefix for a rollout, if available.

```python
nemo_gym.server_utils.set_global_aiohttp_client(
    cfg: nemo_gym.server_utils.GlobalAIOHTTPAsyncClientConfig
) -> aiohttp.ClientSession
```

```python
nemo_gym.server_utils.set_is_nemo_gym_fastapi_entrypoint(
    file: str
) -> None
```

```python
nemo_gym.server_utils.set_is_nemo_gym_fastapi_worker() -> None
```

```python
nemo_gym.server_utils.set_nemo_gym_fastapi_num_workers(
    num_workers: int
) -> None
```

```python
nemo_gym.server_utils.setup_server_client(
    head_server_config: typing.Optional[nemo_gym.config_types.BaseServerConfig] = None
) -> nemo_gym.server_utils.ServerClient
```

```python
nemo_gym.server_utils.DEFAULT_HEAD_SERVER_PORT = 11000
```

```python
nemo_gym.server_utils.DISCONNECTED_CLIENT_OS_HELP_TEXT = "We've run into this issue in two different scenarios previously:\n1. Too many o...
```

```python
nemo_gym.server_utils.DISCONNECTED_CLIENT_OS_PRINT_INTERVAL: int = 100
```

```python
nemo_gym.server_utils.IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME = 'IS_NEMO_GYM_FASTAPI_ENTRYPOINT'
```

```python
nemo_gym.server_utils.IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME = 'IS_NEMO_GYM_FASTAPI_WORKER'
```

```python
nemo_gym.server_utils.MAX_NUM_TRIES = 3
```

```python
nemo_gym.server_utils.NEMO_GYM_FASTAPI_NUM_WORKERS = 'NEMO_GYM_FASTAPI_NUM_WORKERS'
```

```python
nemo_gym.server_utils.NEMO_GYM_MODEL_SERVER_BASE_URL_ENV_VAR_NAME = 'NEMO_GYM_MODEL_SERVER_BASE_URL'
```

```python
nemo_gym.server_utils.NEMO_GYM_MODEL_SERVER_NAME_ENV_VAR_NAME = 'NEMO_GYM_MODEL_SERVER_NAME'
```

```python
nemo_gym.server_utils.SESSION_ID_KEY = 'session_id'
```

```python
nemo_gym.server_utils.ServerStatus = Union[Literal['success'], Literal['connection_error'], Literal['timeout'], Liter...
```

```python
nemo_gym.server_utils._ALL_V4_MAPPED = ip_network('::ffff:0:0/96')
```

```python
nemo_gym.server_utils._GLOBAL_AIOHTTP_CLIENT: Union[None, ClientSession] = None
```

```python
nemo_gym.server_utils._GLOBAL_AIOHTTP_CLIENT_QUEUE_TELEMETRY: bool = False
```

```python
nemo_gym.server_utils._GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG: bool = False
```

```python
nemo_gym.server_utils._NEMO_GYM_STARTED_RAY_CLUSTER: bool = False
```

```python
nemo_gym.server_utils._NUM_CLIENT_OS_ERROR: int = 0
```

```python
nemo_gym.server_utils._NUM_SERVER_DISCONNECTED_ERROR: int = 0
```

```python
nemo_gym.server_utils._TELEMETRY_SERVER_TYPE_BY_BASE = {'SimpleResourcesServer': 'resources_servers', 'SimpleResponsesAPIAgent': 'respo...
```

```python
nemo_gym.server_utils._UPSTREAM_ERROR_LOG_BODY_CHARS = 2000
```

```python
nemo_gym.server_utils._VALIDATION_ERROR_LOG_BODY_CHARS = 4096
```

```python
nemo_gym.server_utils._VALIDATION_ERROR_LOG_FIELD_CHARS = 256
```

```python
nemo_gym.server_utils._VALIDATION_ERROR_LOG_LOC_ITEMS = 8
```

```python
nemo_gym.server_utils._VALIDATION_ERROR_LOG_MAX_ERRORS = 20
```

```python
nemo_gym.server_utils._WARNED_IMPLICIT_RAY_SERVERS: set[type] = set()
```

```python
nemo_gym.server_utils.logger = logging.getLogger(__name__)
```