> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.chat_streaming

Streaming Chat Completions-dialect support shared by every Gym model server.

Blackbox harnesses that speak the OpenAI Chat Completions API over SSE (e.g. the OpenClaw
agent PinchBench runs) send requests the strict `NeMoGymChatCompletionCreateParamsNonStreaming`
model rejects: a `stream: true` flag (its `stream` field is typed `Literal[False]`, since a
Gym model server calls its backend non-streaming and buffers the whole response) plus a
`stream_options` block, which is only meaningful alongside `stream: true`.

A Gym model server can still serve these clients by computing the complete Chat Completion with
its existing non-streaming backend call and re-emitting it as an SSE stream. This module provides:

* the request-side sanitizer that maps the streaming wire body onto the strict params shape
  (drops `stream`/`stream_options`, remembers whether usage was requested);
* the response-side synthesizer that re-emits a complete `NeMoGymChatCompletion` as the
  `chat.completion.chunk` SSE sequence a streaming client expects, terminated by `data: [DONE]`.

Only the SSE envelope is synthesized -- there is no true token-by-token streaming. The dispatch
sends keepalive comments while the backend computes, then emits the complete answer. Backend
failures after the stream starts become terminal SSE errors. External staging stores training
tokens separately; token ids and training logprobs are not carried in the `chat.completion.chunk`
schema. A client that does not set `stream_options.include_usage`
gets no usage chunk, so a model-call record reconstructed from this stream will lack token counts.

## Module Contents

### Functions

| Name                                                                                        | Description                                                                           |
| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| [`_choice_deltas`](#nemo_gym-chat_streaming-_choice_deltas)                                 | Yield the `chat.completion.chunk` choice deltas for one completed choice.             |
| [`_chunk`](#nemo_gym-chat_streaming-_chunk)                                                 | Build one `chat.completion.chunk` object sharing the completion's id/created/model.   |
| [`_sse_data`](#nemo_gym-chat_streaming-_sse_data)                                           | -                                                                                     |
| [`_wants_usage`](#nemo_gym-chat_streaming-_wants_usage)                                     | Whether the client asked for a terminal usage chunk (`stream_options.include_usage`). |
| [`sanitize_streaming_chat_body`](#nemo_gym-chat_streaming-sanitize_streaming_chat_body)     | Map a streaming-dialect chat body onto the strict non-streaming params shape.         |
| [`synthesize_chat_completion_sse`](#nemo_gym-chat_streaming-synthesize_chat_completion_sse) | Re-emit a complete Chat Completion object as a `chat.completion.chunk` SSE stream.    |

### Data

[`LOG`](#nemo_gym-chat_streaming-LOG)

[`_PARAM_FIELDS`](#nemo_gym-chat_streaming-_PARAM_FIELDS)

### API

```python
nemo_gym.chat_streaming._choice_deltas(
    index: int,
    choice: dict[str, typing.Any]
) -> typing.Iterator[dict[str, typing.Any]]
```

Yield the `chat.completion.chunk` choice deltas for one completed choice.

A role delta opens the choice; reasoning, content, refusal, and tool-call deltas follow (each emitted
only when present); a terminal delta carries the `finish_reason`. Splitting the message this
way keeps every field a client tracks (role, reasoning, content, refusal, tool-call name/arguments,
finish reason) in the delta position that client expects, even though it is a single logical
chunk sequence rather than incremental tokens.

```python
nemo_gym.chat_streaming._chunk(
    completion: dict[str, typing.Any],
    choices: list[dict[str, typing.Any]],
    usage: typing.Any = None
) -> dict[str, typing.Any]
```

Build one `chat.completion.chunk` object sharing the completion's id/created/model.

```python
nemo_gym.chat_streaming._sse_data(
    payload: dict[str, typing.Any]
) -> str
```

```python
nemo_gym.chat_streaming._wants_usage(
    stream_options: typing.Any
) -> bool
```

Whether the client asked for a terminal usage chunk (`stream_options.include_usage`).

```python
nemo_gym.chat_streaming.sanitize_streaming_chat_body(
    body: dict[str, typing.Any]
) -> tuple[dict[str, typing.Any], bool]
```

Map a streaming-dialect chat body onto the strict non-streaming params shape.

Returns the cleaned body dict (ready for `NeMoGymChatCompletionCreateParamsNonStreaming`
validation) and whether a terminal usage chunk was requested via
`stream_options.include_usage`.

`stream` and `stream_options` are removed: the params model's `stream` field is typed
`Literal[False]`, and `stream_options` is only meaningful with `stream: true`, so it has
no effect on the non-streaming backend call. Remaining fields are filtered to the known params
fields, so a harness's extra bookkeeping never reaches the backend.

```python
nemo_gym.chat_streaming.synthesize_chat_completion_sse(
    completion: dict[str, typing.Any],
    include_usage: bool = False
) -> typing.Iterator[str]
```

Re-emit a complete Chat Completion object as a `chat.completion.chunk` SSE stream.

Emits, per choice, a role chunk -> optional reasoning/content/refusal/tool-call chunks -> a terminal
chunk carrying `finish_reason`. When `include_usage` is set and the completion reports
usage, a final `choices: []` chunk carries the usage block (OpenAI's contract). The stream
always ends with the `data: [DONE]` sentinel streaming clients treat as terminal.

```python
nemo_gym.chat_streaming.LOG = logging.getLogger(__name__)
```

```python
nemo_gym.chat_streaming._PARAM_FIELDS = frozenset(NeMoGymChatCompletionCreateParamsNonStreaming.model_fields) - CHAT_REQ...
```