> This page is for version 0.3.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.models.base_responses_api_model

Model server base classes.

Every Gym model server derives from `SimpleResponsesAPIModel`.
It serves the three model dialects: /v1/responses, /v1/chat/completions, and /v1/messages.
It also installs the model-call capture middleware from `nemo_gym.models.model_call_capture`.

## Module Contents

### Classes

| Name                                                                                                   | Description                                   |
| ------------------------------------------------------------------------------------------------------ | --------------------------------------------- |
| [`BaseResponsesAPIModel`](#nemo_gym-models-base_responses_api_model-BaseResponsesAPIModel)             | -                                             |
| [`BaseResponsesAPIModelConfig`](#nemo_gym-models-base_responses_api_model-BaseResponsesAPIModelConfig) | -                                             |
| [`ModelCheckpointConfig`](#nemo_gym-models-base_responses_api_model-ModelCheckpointConfig)             | A model server instance's `checkpoint` block. |
| [`SimpleResponsesAPIModel`](#nemo_gym-models-base_responses_api_model-SimpleResponsesAPIModel)         | -                                             |

### Functions

| Name                                                                                                       | Description                                                                              |
| ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| [`__getattr__`](#nemo_gym-models-base_responses_api_model-__getattr__)                                     | -                                                                                        |
| [`_orjson_dispatch_response`](#nemo_gym-models-base_responses_api_model-_orjson_dispatch_response)         | Serialize a completed non-streaming model response with orjson.                          |
| [`_plain`](#nemo_gym-models-base_responses_api_model-_plain)                                               | Reduce a request field to plain data so equal schemas serialize equally.                 |
| [`_request_envelope`](#nemo_gym-models-base_responses_api_model-_request_envelope)                         | Return prompt-shaping request fields that are not turns.                                 |
| [`_request_messages`](#nemo_gym-models-base_responses_api_model-_request_messages)                         | Return the conversation carried by any supported dialect.                                |
| [`_validate_chat_params`](#nemo_gym-models-base_responses_api_model-_validate_chat_params)                 | Validate a /v1/chat/completions body dict, surfacing failures as FastAPI's standard 422. |
| [`_validate_responses_params`](#nemo_gym-models-base_responses_api_model-_validate_responses_params)       | Validate a /v1/responses body dict, surfacing failures as FastAPI's standard 422.        |
| [`context_overflow_error_fields`](#nemo_gym-models-base_responses_api_model-context_overflow_error_fields) | Return `(message, code, param)` for a marked engine refusal.                             |
| [`is_propagated_refusal`](#nemo_gym-models-base_responses_api_model-is_propagated_refusal)                 | Whether a model server marked `error` as a refusal it propagates on purpose.             |

### Data

[`PROPAGATED_REFUSAL_CODE_ATTRIBUTE`](#nemo_gym-models-base_responses_api_model-PROPAGATED_REFUSAL_CODE_ATTRIBUTE)

[`_ANTHROPIC_CONVERTER`](#nemo_gym-models-base_responses_api_model-_ANTHROPIC_CONVERTER)

[`_CHAT_KEEPALIVE_SECONDS`](#nemo_gym-models-base_responses_api_model-_CHAT_KEEPALIVE_SECONDS)

[`_ENVELOPE_ROLE`](#nemo_gym-models-base_responses_api_model-_ENVELOPE_ROLE)

[`_moved_lazy_names`](#nemo_gym-models-base_responses_api_model-_moved_lazy_names)

[`_moved_names`](#nemo_gym-models-base_responses_api_model-_moved_names)

[`logger`](#nemo_gym-models-base_responses_api_model-logger)

### API

```python
class nemo_gym.models.base_responses_api_model.BaseResponsesAPIModel()
```

**Bases:** [BaseServer](/nemo/gym/nemo-gym/nemo_gym/server_runtime/base_server#nemo_gym-server_runtime-base_server-BaseServer)

**`config`** `BaseResponsesAPIModelConfig`

---

```python
class nemo_gym.models.base_responses_api_model.BaseResponsesAPIModelConfig()
```

**Bases:** [BaseRunServerInstanceConfig](/nemo/gym/nemo-gym/nemo_gym/config/config_types#nemo_gym-config-config_types-BaseRunServerInstanceConfig)

**`checkpoint`** `ModelCheckpointConfig = Field(default_factory=ModelCheckpointConfig)`

---

**`custom_tool_strategy`** `CustomToolStrategy`

---

**`drop_custom_tools`** `bool`

---

**`drop_hosted_tools`** `bool`

---

**`stream_keepalive_s`** `Optional[float]`

---

**`token_id_capture_non_generating_requests`** `list[NonGeneratingRequest] = Field(default_factory=list)`

---

```python
class nemo_gym.models.base_responses_api_model.ModelCheckpointConfig()
```

**Bases:** `BaseModel`

A model server instance's `checkpoint` block.

**`generation_cuts`** `bool = False`

---

**`model_config`** `= ConfigDict(extra='forbid')`

---

**`policy`** `bool = False`

---

```python
class nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel()
```

**Bases:** [BaseResponsesAPIModel](#nemo_gym-models-base_responses_api_model-BaseResponsesAPIModel), [SimpleServer](/nemo/gym/nemo-gym/nemo_gym/server_runtime/base_server#nemo_gym-server_runtime-base_server-SimpleServer)

**`_capture_lineage`** `LineageResolver | None = PrivateAttr(default=None)`

---

**`_policy_coordinator`** `Optional[PolicyCoordinator] = PrivateAttr(default=None)`

---

**`non_generating_model_routes`** `frozenset[tuple[str, str]] = frozenset()`

---

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._finalize_served_response(
    response: typing.Any
) -> None
```

async

Finalize capture after conversion to the response returned to the client.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._invoke_chat_completions(
    request: fastapi.Request,
    params: nemo_gym.core.responses_api_types.NeMoGymChatCompletionCreateParamsNonStreaming
) -> nemo_gym.core.responses_api_types.NeMoGymChatCompletion
```

async

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._invoke_responses(
    request: fastapi.Request,
    params: nemo_gym.core.responses_api_types.NeMoGymResponseCreateParamsNonStreaming,
    response_id: typing.Optional[str] = None
) -> nemo_gym.core.responses_api_types.NeMoGymResponse
```

async

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._mints_response_ids() -> bool
```

Whether the ids of the responses `responses()` returns are this server's own.

A keepalive stream (`stream_keepalive_s`) names its response before `responses()` returns,
so that response takes the announced id. An id an upstream Responses API issued must reach the
client unchanged, so a server that serves those cannot keep a stream alive.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._responses_keepalive_events(
    request: fastapi.Request,
    pending: asyncio.Task[typing.Any],
    ns_map: nemo_gym.models.responses_streaming.NamespaceMap,
    keepalive_s: float,
    response_id: str,
    failure_log: typing.Optional[nemo_gym.core.failure_transport.RequestFailureLog] = None
) -> collections.abc.AsyncIterator[str | bytes]
```

async

Keep an opened Responses stream alive until `pending`, which serves `response_id`, completes.

The headers went out before `pending` finished, so the terminal event carries the request's failure report.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._sanitize_streaming_responses_body(
    body: dict
) -> tuple[dict, nemo_gym.models.responses_streaming.NamespaceMap]
```

Apply this server's tool policy while mapping a streaming body to the strict shape.

Every `/v1/responses` dispatch that accepts the streaming wire dialect sanitizes
through here, so the config's `drop_custom_tools`, `drop_hosted_tools` and
`custom_tool_strategy` settings reach the sanitizer on each of them.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._serve_responses_sse(
    request: fastapi.Request,
    response: typing.Any,
    ns_map: nemo_gym.models.responses_streaming.NamespaceMap,
    include_created: bool = True
) -> fastapi.responses.StreamingResponse
```

async

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._should_propagate_upstream_http_error(
    error: aiohttp.ClientResponseError
) -> bool
```

Allow adapters to retain an explicit upstream-status compatibility policy.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._stream_buffered_responses(
    request: fastapi.Request,
    invocation: collections.abc.Awaitable[typing.Any],
    ns_map: nemo_gym.models.responses_streaming.NamespaceMap
) -> fastapi.responses.StreamingResponse
```

async

Stream a completed response; until it completes nothing is sent to the client.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._stream_responses(
    request: fastapi.Request,
    params: nemo_gym.core.responses_api_types.NeMoGymResponseCreateParamsNonStreaming,
    ns_map: nemo_gym.models.responses_streaming.NamespaceMap
) -> fastapi.responses.StreamingResponse
```

async

Serve the same converted response to capture and Responses SSE clients.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._stream_served_response(
    response: typing.Any,
    events: typing.Iterable[str | bytes]
) -> fastapi.responses.StreamingResponse
```

async

Serialize buffered SSE before committing externally staged capture.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._streamed_responses_failure(
    exc: Exception
) -> tuple[typing.Optional[str], str]
```

The `(code, message)` of the `response.failed` event that reports a failure after keepalives.

Mirrors what the client would have seen before the stream opened: a refusal the server
propagates keeps its OpenAI error code (`context_length_exceeded` tells a harness to compact
and retry), and an upstream HTTP error follows the server's propagation policy. Upstream
errors never report their request URL.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel._validate_capture_request(
    body: nemo_gym.core.responses_api_types.NeMoGymResponseCreateParamsNonStreaming | nemo_gym.core.responses_api_types.NeMoGymChatCompletionCreateParamsNonStreaming
) -> None
```

Validate server-specific request options before recording capture intent.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.chat_completions(
    body: nemo_gym.core.responses_api_types.NeMoGymChatCompletionCreateParamsNonStreaming = Body()
) -> nemo_gym.core.responses_api_types.NeMoGymChatCompletion
```

async

abstract

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.chat_completions_dispatch(
    request: fastapi.Request,
    body: dict = Body()
)
```

async

Default `/v1/chat/completions` entrypoint shared by every Gym model server.

A non-streaming request validates strictly against
`NeMoGymChatCompletionCreateParamsNonStreaming` and delegates to this server's own
`chat_completions()`, preserving the historical non-streaming behavior (including the
standard 422 shape). When the client sends `stream: true` (blackbox
Chat-Completions-over-SSE harnesses like the OpenClaw agent always do), the request is
sanitized onto that same strict model (drop `stream`/`stream_options`; see
`nemo_gym.models.chat_streaming`), validated identically, delegated to the same
`chat_completions()`, and the complete response is buffered and re-emitted as a
synthesized `chat.completion.chunk` SSE stream. Keepalive comments maintain the
connection while the backend computes; model output is still buffer-then-replay.

Only a genuine boolean `stream: true` takes the streaming path; any other value
(e.g. `"false"` or `1`) stays on the strict non-streaming path, which rejects the
malformed `stream` with the same 422 as before.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.embeddings(
    body: dict = Body()
) -> dict
```

async

OpenAI-compatible `POST /v1/embeddings` default: 501 unless a server overrides it.

Servers backed by an embeddings-capable provider (e.g. `openai_model`,
`inference_provider`) override this to pass `body` through to the provider and
return the provider's JSON unchanged.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.generation_cut_control_root(
    base_url: str
) -> str
```

The URL root where the inference worker serving `base_url` takes generation cuts.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.messages(
    request: fastapi.Request,
    body: dict = Body()
)
```

async

Default Anthropic Messages \<-> Responses mapping shared by every Gym model server.

Translates the inbound Anthropic Messages request to the Responses API, delegates to this
server's own `responses()` (so it reuses whatever backend the server has), and maps the
result back to an Anthropic Messages response. When the client requested `stream: true`
(the Claude Code CLI always does), the complete response is re-emitted as a synthesized
Anthropic SSE event stream. Servers may override this for native Messages handling.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.responses(
    body: nemo_gym.core.responses_api_types.NeMoGymResponseCreateParamsNonStreaming = Body()
) -> nemo_gym.core.responses_api_types.NeMoGymResponse
```

async

abstract

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.responses_dispatch(
    request: fastapi.Request,
    body: dict = Body()
)
```

async

Default `/v1/responses` entrypoint shared by every Gym model server.

A plain JSON request validates strictly against
`NeMoGymResponseCreateParamsNonStreaming` and delegates to this server's own
`responses()`, preserving the historical non-streaming behavior. When the client
requests `stream: true` (blackbox Responses-over-SSE harnesses like the Codex CLI
always do), the request is first sanitized from the streaming wire dialect (extra
bookkeeping fields, `namespace` tool specs — see `nemo_gym.models.responses_streaming`),
delegated to the same `responses()`, and the complete response is re-emitted as a
synthesized Responses SSE event stream. Upstream HTTP errors and `HTTPException`
retain their status before the stream starts. Other `responses()` failures become terminal
`response.failed` events; bad-request validation still fails eagerly. With
`stream_keepalive_s` set, a call that outlasts it opens the stream early and keeps it alive
with `response.in_progress` events, and any later failure becomes `response.failed`.

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.setup_policy_checkpoint(
    app: fastapi.FastAPI,
    capture_ledger: nemo_gym.models.token_capture.protocols.CaptureLedger | None
) -> None
```

```python
nemo_gym.models.base_responses_api_model.SimpleResponsesAPIModel.setup_webserver() -> fastapi.FastAPI
```

```python
nemo_gym.models.base_responses_api_model.__getattr__(
    name: str
) -> typing.Any
```

```python
nemo_gym.models.base_responses_api_model._orjson_dispatch_response(
    content: typing.Any
) -> typing.Any
```

Serialize a completed non-streaming model response with orjson.

FastAPI serializes a bare dictionary or Pydantic model with `jsonable_encoder`.
That function recursively visits every token ID and log probability before standard-library `json.dumps` runs.

Convert Pydantic models to JSON-compatible values before encoding the result.
Return the encoded bytes as a `Response` so FastAPI does not encode them again.
Preserve `Response` objects created by model-server overrides.

```python
nemo_gym.models.base_responses_api_model._plain(
    value: typing.Any
) -> typing.Any
```

Reduce a request field to plain data so equal schemas serialize equally.

Handlers can expose tools as dictionaries or Pydantic models.
Normalization keeps an unchanged schema stable across handlers.

```python
nemo_gym.models.base_responses_api_model._request_envelope(
    getter: typing.Any
) -> list[dict]
```

Return prompt-shaping request fields that are not turns.

Instructions and tools can be siblings of the message list.
The chat template renders both into the prompt.
Including them prevents prefix reuse across different request envelopes.

```python
nemo_gym.models.base_responses_api_model._request_messages(
    body: typing.Any
) -> list[dict]
```

Return the conversation carried by any supported dialect.

Lineage uses model-authored turns to identify the parent call.
Chat and Anthropic use `messages`.
Responses uses `input`.
The request envelope is prepended as a pseudo-turn because it also shapes the prompt.

```python
nemo_gym.models.base_responses_api_model._validate_chat_params(
    body: dict
) -> nemo_gym.core.responses_api_types.NeMoGymChatCompletionCreateParamsNonStreaming
```

Validate a /v1/chat/completions body dict, surfacing failures as FastAPI's standard 422.

`include_url=False` mirrors FastAPI's native body validation, which strips the
`errors.pydantic.dev` url from each detail entry, so the 422 body stays byte-for-byte
identical to the previous typed-`Body()` binding.

```python
nemo_gym.models.base_responses_api_model._validate_responses_params(
    body: dict
) -> nemo_gym.core.responses_api_types.NeMoGymResponseCreateParamsNonStreaming
```

Validate a /v1/responses body dict, surfacing failures as FastAPI's standard 422.

```python
nemo_gym.models.base_responses_api_model.context_overflow_error_fields(
    error: BaseException
) -> tuple[str, str | None, str | None]
```

Return `(message, code, param)` for a marked engine refusal.

The engine's error body travels on the exception as `response_content` (attached by
`nemo_gym.client.http_client.raise_for_status`). vLLM 0.10.1 and later format it as
`{"error": {"message": ..., "type": ..., "param": ..., "code": 400}}`; older engines used a
flat object with a top-level `message`. The message is the engine's own sentence from
either shape, verbatim; any other body is returned as raw text so no information is lost.
The param is the engine's `param` when its body names one. The code is whatever the server
stored under `PROPAGATED_REFUSAL_CODE_ATTRIBUTE` when it marked the error.

```python
nemo_gym.models.base_responses_api_model.is_propagated_refusal(
    error: BaseException
) -> bool
```

Whether a model server marked `error` as a refusal it propagates on purpose.

```python
nemo_gym.models.base_responses_api_model.PROPAGATED_REFUSAL_CODE_ATTRIBUTE = 'nemo_gym_propagated_refusal_code'
```

```python
nemo_gym.models.base_responses_api_model._ANTHROPIC_CONVERTER = AnthropicConverter()
```

```python
nemo_gym.models.base_responses_api_model._CHAT_KEEPALIVE_SECONDS = 15.0
```

```python
nemo_gym.models.base_responses_api_model._ENVELOPE_ROLE = '_ng_request_envelope'
```

```python
nemo_gym.models.base_responses_api_model._moved_lazy_names = moved_attr_getter(__name__, {'token_id_capture_config': 'nemo_gym.models.token_c...
```

```python
nemo_gym.models.base_responses_api_model._moved_names = moved_names_getattr(__name__)
```

```python
nemo_gym.models.base_responses_api_model.logger = logging.getLogger(__name__)
```