> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.server_utils

## Module Contents

### Classes

| Name                                                                                      | Description                                                   |
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| [`BaseServer`](#nemo_gym-server_utils-BaseServer)                                         | All instances of BaseServer are queryable using ServerClient. |
| [`GlobalAIOHTTPAsyncClientConfig`](#nemo_gym-server_utils-GlobalAIOHTTPAsyncClientConfig) | -                                                             |
| [`HeadServer`](#nemo_gym-server_utils-HeadServer)                                         | -                                                             |
| [`ProfilingMiddlewareConfig`](#nemo_gym-server_utils-ProfilingMiddlewareConfig)           | -                                                             |
| [`ProfilingMiddlewareInputConfig`](#nemo_gym-server_utils-ProfilingMiddlewareInputConfig) | -                                                             |
| [`ServerClient`](#nemo_gym-server_utils-ServerClient)                                     | -                                                             |
| [`ServerInstanceDisplayConfig`](#nemo_gym-server_utils-ServerInstanceDisplayConfig)       | -                                                             |
| [`SimpleServer`](#nemo_gym-server_utils-SimpleServer)                                     | -                                                             |
| [`UvicornLoggingConfig`](#nemo_gym-server_utils-UvicornLoggingConfig)                     | -                                                             |

### Functions

| Name                                                                                                                      | Description                          |
| ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
| [`get_global_aiohttp_client`](#nemo_gym-server_utils-get_global_aiohttp_client)                                           | -                                    |
| [`get_nemo_gym_fastapi_num_workers`](#nemo_gym-server_utils-get_nemo_gym_fastapi_num_workers)                             | -                                    |
| [`get_response_json`](#nemo_gym-server_utils-get_response_json)                                                           | -                                    |
| [`get_server_url`](#nemo_gym-server_utils-get_server_url)                                                                 | -                                    |
| [`global_aiohttp_client_exit`](#nemo_gym-server_utils-global_aiohttp_client_exit)                                         | -                                    |
| [`initialize_ray`](#nemo_gym-server_utils-initialize_ray)                                                                 | Initialize ray cluster in a process. |
| [`is_global_aiohttp_client_request_debug_enabled`](#nemo_gym-server_utils-is_global_aiohttp_client_request_debug_enabled) | -                                    |
| [`is_global_aiohttp_client_setup`](#nemo_gym-server_utils-is_global_aiohttp_client_setup)                                 | -                                    |
| [`is_nemo_gym_fastapi_entrypoint`](#nemo_gym-server_utils-is_nemo_gym_fastapi_entrypoint)                                 | -                                    |
| [`is_nemo_gym_fastapi_worker`](#nemo_gym-server_utils-is_nemo_gym_fastapi_worker)                                         | -                                    |
| [`maybe_ray_cluster_exit`](#nemo_gym-server_utils-maybe_ray_cluster_exit)                                                 | -                                    |
| [`raise_for_status`](#nemo_gym-server_utils-raise_for_status)                                                             | -                                    |
| [`request`](#nemo_gym-server_utils-request)                                                                               | -                                    |
| [`set_global_aiohttp_client`](#nemo_gym-server_utils-set_global_aiohttp_client)                                           | -                                    |
| [`set_is_nemo_gym_fastapi_entrypoint`](#nemo_gym-server_utils-set_is_nemo_gym_fastapi_entrypoint)                         | -                                    |
| [`set_is_nemo_gym_fastapi_worker`](#nemo_gym-server_utils-set_is_nemo_gym_fastapi_worker)                                 | -                                    |
| [`set_nemo_gym_fastapi_num_workers`](#nemo_gym-server_utils-set_nemo_gym_fastapi_num_workers)                             | -                                    |

### Data

[`DEFAULT_HEAD_SERVER_PORT`](#nemo_gym-server_utils-DEFAULT_HEAD_SERVER_PORT)

[`DISCONNECTED_CLIENT_OS_HELP_TEXT`](#nemo_gym-server_utils-DISCONNECTED_CLIENT_OS_HELP_TEXT)

[`DISCONNECTED_CLIENT_OS_PRINT_INTERVAL`](#nemo_gym-server_utils-DISCONNECTED_CLIENT_OS_PRINT_INTERVAL)

[`IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME`](#nemo_gym-server_utils-IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME)

[`IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME`](#nemo_gym-server_utils-IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME)

[`MAX_NUM_TRIES`](#nemo_gym-server_utils-MAX_NUM_TRIES)

[`NEMO_GYM_FASTAPI_NUM_WORKERS`](#nemo_gym-server_utils-NEMO_GYM_FASTAPI_NUM_WORKERS)

[`SESSION_ID_KEY`](#nemo_gym-server_utils-SESSION_ID_KEY)

[`ServerStatus`](#nemo_gym-server_utils-ServerStatus)

[`_GLOBAL_AIOHTTP_CLIENT`](#nemo_gym-server_utils-_GLOBAL_AIOHTTP_CLIENT)

[`_GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG`](#nemo_gym-server_utils-_GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG)

[`_NEMO_GYM_STARTED_RAY_CLUSTER`](#nemo_gym-server_utils-_NEMO_GYM_STARTED_RAY_CLUSTER)

[`_NUM_CLIENT_OS_ERROR`](#nemo_gym-server_utils-_NUM_CLIENT_OS_ERROR)

[`_NUM_SERVER_DISCONNECTED_ERROR`](#nemo_gym-server_utils-_NUM_SERVER_DISCONNECTED_ERROR)

### API

```python
class nemo_gym.server_utils.BaseServer()
```

**Bases:** `BaseModel`

All instances of BaseServer are queryable using ServerClient.

```python
nemo_gym.server_utils.BaseServer.load_config_from_global_config() -> nemo_gym.config_types.BaseRunServerInstanceConfig
```

classmethod

```python
nemo_gym.server_utils.BaseServer.setup_liveness(
    app: fastapi.FastAPI
) -> None
```

```python
class nemo_gym.server_utils.GlobalAIOHTTPAsyncClientConfig()
```

**Bases:** `BaseModel`

```python
class nemo_gym.server_utils.HeadServer()
```

**Bases:** [BaseServer](#nemo_gym-server_utils-BaseServer)

```python
nemo_gym.server_utils.HeadServer.get_server_instances() -> typing.List[dict]
```

```python
nemo_gym.server_utils.HeadServer.global_config_dict_yaml() -> str
```

async

```python
nemo_gym.server_utils.HeadServer.run_webserver() -> typing.Tuple[uvicorn.Server, threading.Thread, nemo_gym.server_utils.HeadServer]
```

classmethod

```python
nemo_gym.server_utils.HeadServer.set_server_instances(
    instances: typing.List
) -> None
```

```python
nemo_gym.server_utils.HeadServer.setup_webserver() -> fastapi.FastAPI
```

```python
class nemo_gym.server_utils.ProfilingMiddlewareConfig()
```

**Bases:** [ProfilingMiddlewareInputConfig](#nemo_gym-server_utils-ProfilingMiddlewareInputConfig)

```python
class nemo_gym.server_utils.ProfilingMiddlewareInputConfig()
```

**Bases:** `BaseModel`

```python
class nemo_gym.server_utils.ServerClient()
```

**Bases:** `BaseModel`

```python
nemo_gym.server_utils.ServerClient._build_server_base_url(
    server_config_dict: omegaconf.OmegaConf
) -> str
```

```python
nemo_gym.server_utils.ServerClient.get(
    server_name: str,
    url_path: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

**Parameters:**

str
The name of the server you are trying to call.

str
The URL path in the server you are trying to call e.g. "/v1/responses".

```python
nemo_gym.server_utils.ServerClient.load_from_global_config(
    head_server_config: typing.Optional[nemo_gym.config_types.BaseServerConfig] = None
) -> nemo_gym.server_utils.ServerClient
```

classmethod

```python
nemo_gym.server_utils.ServerClient.load_head_server_config() -> nemo_gym.config_types.BaseServerConfig
```

classmethod

```python
nemo_gym.server_utils.ServerClient.poll_for_status(
    server_name: str
) -> nemo_gym.server_utils.ServerStatus
```

```python
nemo_gym.server_utils.ServerClient.post(
    server_name: str,
    url_path: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

**Parameters:**

str
The name of the server you are trying to call.

str
The URL path in the server you are trying to call e.g. "/v1/responses".

```python
nemo_gym.server_utils.ServerClient.request(
    server_name: str,
    url_path: str,
    method: str,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

```python
class nemo_gym.server_utils.ServerInstanceDisplayConfig()
```

**Bases:** `BaseModel`

```python
class nemo_gym.server_utils.SimpleServer()
```

**Bases:** [BaseServer](#nemo_gym-server_utils-BaseServer)

```python
nemo_gym.server_utils.SimpleServer.get_session_middleware_key() -> str
```

```python
nemo_gym.server_utils.SimpleServer.prefix_server_logs() -> None
```

```python
nemo_gym.server_utils.SimpleServer.run_webserver() -> typing.Optional[fastapi.FastAPI]
```

classmethod

```python
nemo_gym.server_utils.SimpleServer.set_ulimit(
    target_soft_limit: int = 65535
)
```

```python
nemo_gym.server_utils.SimpleServer.setup_exception_middleware(
    app: fastapi.FastAPI
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_profiling(
    app: fastapi.FastAPI,
    profiling_config: nemo_gym.server_utils.ProfilingMiddlewareConfig
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_session_middleware(
    app: fastapi.FastAPI
) -> None
```

```python
nemo_gym.server_utils.SimpleServer.setup_webserver() -> fastapi.FastAPI
```

abstract

```python
class nemo_gym.server_utils.UvicornLoggingConfig()
```

**Bases:** `BaseModel`

```python
nemo_gym.server_utils.get_global_aiohttp_client(
    global_config_dict_parser_config: typing.Optional[nemo_gym.global_config.GlobalConfigDictParserConfig] = None,
    global_config_dict_parser_cls: typing.Type[nemo_gym.global_config.GlobalConfigDictParser] = GlobalConfigDictParser
) -> aiohttp.ClientSession
```

```python
nemo_gym.server_utils.get_nemo_gym_fastapi_num_workers() -> int
```

```python
nemo_gym.server_utils.get_response_json(
    response: aiohttp.ClientResponse
) -> typing.Any
```

async

```python
nemo_gym.server_utils.get_server_url(
    server_name: str
) -> str
```

```python
nemo_gym.server_utils.global_aiohttp_client_exit()
```

```python
nemo_gym.server_utils.initialize_ray() -> None
```

Initialize ray cluster in a process.
We store the Ray address in the global config dict so that child processes can connect to it.
This avoids the need to start a new Ray cluster in each child process.
Note: This function will modify the global config dict - update `ray_head_node_address`

```python
nemo_gym.server_utils.is_global_aiohttp_client_request_debug_enabled() -> bool
```

```python
nemo_gym.server_utils.is_global_aiohttp_client_setup() -> bool
```

```python
nemo_gym.server_utils.is_nemo_gym_fastapi_entrypoint(
    file: str
) -> bool
```

```python
nemo_gym.server_utils.is_nemo_gym_fastapi_worker() -> bool
```

```python
nemo_gym.server_utils.maybe_ray_cluster_exit()
```

```python
nemo_gym.server_utils.raise_for_status(
    response: aiohttp.ClientResponse
) -> None
```

async

```python
nemo_gym.server_utils.request(
    method: str,
    url: str,
    _internal: bool = False,
    kwargs: typing.Unpack[aiohttp.client._RequestOptions] = {}
) -> aiohttp.ClientResponse
```

async

```python
nemo_gym.server_utils.set_global_aiohttp_client(
    cfg: nemo_gym.server_utils.GlobalAIOHTTPAsyncClientConfig
) -> aiohttp.ClientSession
```

```python
nemo_gym.server_utils.set_is_nemo_gym_fastapi_entrypoint(
    file: str
) -> None
```

```python
nemo_gym.server_utils.set_is_nemo_gym_fastapi_worker() -> None
```

```python
nemo_gym.server_utils.set_nemo_gym_fastapi_num_workers(
    num_workers: int
) -> None
```

```python
nemo_gym.server_utils.DEFAULT_HEAD_SERVER_PORT = 11000
```

```python
nemo_gym.server_utils.DISCONNECTED_CLIENT_OS_HELP_TEXT = "We've run into this issue in two different scenarios previously:\n1. Too many o...
```

```python
nemo_gym.server_utils.DISCONNECTED_CLIENT_OS_PRINT_INTERVAL: int = 100
```

```python
nemo_gym.server_utils.IS_NEMO_GYM_FASTAPI_ENTRYPOINT_KEY_NAME = 'IS_NEMO_GYM_FASTAPI_ENTRYPOINT'
```

```python
nemo_gym.server_utils.IS_NEMO_GYM_FASTAPI_WORKER_KEY_NAME = 'IS_NEMO_GYM_FASTAPI_WORKER'
```

```python
nemo_gym.server_utils.MAX_NUM_TRIES = 3
```

```python
nemo_gym.server_utils.NEMO_GYM_FASTAPI_NUM_WORKERS = 'NEMO_GYM_FASTAPI_NUM_WORKERS'
```

```python
nemo_gym.server_utils.SESSION_ID_KEY = 'session_id'
```

```python
nemo_gym.server_utils.ServerStatus = Union[Literal['success'], Literal['connection_error'], Literal['timeout'], Liter...
```

```python
nemo_gym.server_utils._GLOBAL_AIOHTTP_CLIENT: Union[None, ClientSession] = None
```

```python
nemo_gym.server_utils._GLOBAL_AIOHTTP_CLIENT_REQUEST_DEBUG: bool = False
```

```python
nemo_gym.server_utils._NEMO_GYM_STARTED_RAY_CLUSTER: bool = False
```

```python
nemo_gym.server_utils._NUM_CLIENT_OS_ERROR: int = 0
```

```python
nemo_gym.server_utils._NUM_SERVER_DISCONNECTED_ERROR: int = 0
```