> This page is for version 0.5.1.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.

# nemo_gym.core.model_call_sampling

Sampling values that a training framework sets for a rollout's model calls, and how they reach each call.

A training framework that serves the policy samples the calls of the agents it trains with its own values: its
training values, or its validation values for a validation rollout. It sets them on each rollout's row under
`_ng_model_sampling`, next to `_ng_rollout_id`. Gym carries them into the URL of every model call the rollout's
agent makes, as `/ng-sampling/<values>` after any other prefix, and the Model Server writes them into the request.
Gym never interprets them.

In such a run, a call from a caller the framework does not train, such as a verifier's LLM judge, carries
`/ng-sampling/null`: it keeps the sampling its caller chose, and its own Model Server's `sampling_overrides`.
A run whose rows carry no values, such as Gym-only evaluation, adds no segment.

## Module Contents

### Classes

| Name                                                                                    | Description                                                                                      |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [`ModelCallSampling`](#nemo_gym-core-model_call_sampling-ModelCallSampling)             | Sampling values for the model calls of an agent that a training framework trains.                |
| [`ModelCallSamplingConfig`](#nemo_gym-core-model_call_sampling-ModelCallSamplingConfig) | The global `model_call_sampling` block. A training framework sets it when its rows carry values. |

### Functions

| Name                                                                                                  | Description                                                                                                      |
| ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| [`_unique_keys`](#nemo_gym-core-model_call_sampling-_unique_keys)                                     | -                                                                                                                |
| [`current_model_sampling_segment`](#nemo_gym-core-model_call_sampling-current_model_sampling_segment) | Return the segment value for this request's model calls, or `None` when they carry no segment.                   |
| [`decode_model_sampling`](#nemo_gym-core-model_call_sampling-decode_model_sampling)                   | Decode a segment value: `null` to `None`, anything else to values.                                               |
| [`encode_model_sampling`](#nemo_gym-core-model_call_sampling-encode_model_sampling)                   | Encode values as the segment's unpadded base64url JSON, and `None` as `null`.                                    |
| [`model_call_sampling_config`](#nemo_gym-core-model_call_sampling-model_call_sampling_config)         | Return the run's `model_call_sampling` block, or `None` when the run sets none.                                  |
| [`model_sampling_context`](#nemo_gym-core-model_call_sampling-model_sampling_context)                 | Give the model calls made inside the block the segment value `segment`.                                          |
| [`model_sampling_from_run_body`](#nemo_gym-core-model_call_sampling-model_sampling_from_run_body)     | Return the values a run request or row carries under `_ng_model_sampling`, or `None`.                            |
| [`model_sampling_path_segment`](#nemo_gym-core-model_call_sampling-model_sampling_path_segment)       | Return `/ng-sampling/&lt;segment&gt;`, or `""` for no segment.                                                   |
| [`with_model_sampling_segment`](#nemo_gym-core-model_call_sampling-with_model_sampling_segment)       | Insert `/ng-sampling/&lt;segment&gt;` into a Model Server generation path, where its correlation prefix puts it. |

### Data

[`MODEL_SAMPLING_HEADER`](#nemo_gym-core-model_call_sampling-MODEL_SAMPLING_HEADER)

[`MODEL_SAMPLING_NULL`](#nemo_gym-core-model_call_sampling-MODEL_SAMPLING_NULL)

[`_ENCODED`](#nemo_gym-core-model_call_sampling-_ENCODED)

[`_GENERATION_API_PATHS`](#nemo_gym-core-model_call_sampling-_GENERATION_API_PATHS)

[`_MAX_ENCODED_LENGTH`](#nemo_gym-core-model_call_sampling-_MAX_ENCODED_LENGTH)

[`_MODEL_SAMPLING_SEGMENT`](#nemo_gym-core-model_call_sampling-_MODEL_SAMPLING_SEGMENT)

### API

```python
class nemo_gym.core.model_call_sampling.ModelCallSampling()
```

**Bases:** `BaseModel`

Sampling values for the model calls of an agent that a training framework trains.

The Model Server sends the smaller of `max_output_tokens` and the call's own limit, so a harness that reserves
fewer tokens, near the end of its context or for a summary, keeps its limit. An unset field leaves the call's own
value.

**`max_output_tokens`** `Optional[int] = Field(default=None, gt=0, le=(2 ** 53 - 1))`

---

**`model_config`**

---

**`temperature`** `Optional[float]`

---

**`top_p`** `Optional[float]`

---

```python
class nemo_gym.core.model_call_sampling.ModelCallSamplingConfig()
```

**Bases:** `BaseModel`

The global `model_call_sampling` block. A training framework sets it when its rows carry values.

Its presence marks the run: resources servers then send `null` with the model calls they make through Gym's
server client while handling a request, so their judges keep their own sampling.

**`model_config`** `= ConfigDict(extra='forbid')`

---

**`required_model_servers`** `list[str] = Field(default_factory=list)`

---

```python
nemo_gym.core.model_call_sampling._unique_keys(
    pairs: list[tuple[str, typing.Any]]
) -> dict[str, typing.Any]
```

```python
nemo_gym.core.model_call_sampling.current_model_sampling_segment() -> typing.Optional[str]
```

Return the segment value for this request's model calls, or `None` when they carry no segment.

```python
nemo_gym.core.model_call_sampling.decode_model_sampling(
    encoded: str
) -> typing.Optional[nemo_gym.core.model_call_sampling.ModelCallSampling]
```

Decode a segment value: `null` to `None`, anything else to values.

Raises `ValueError` for a malformed or ambiguous value, so a call never runs with values it did not send.

```python
nemo_gym.core.model_call_sampling.encode_model_sampling(
    sampling: typing.Optional[nemo_gym.core.model_call_sampling.ModelCallSampling]
) -> str
```

Encode values as the segment's unpadded base64url JSON, and `None` as `null`.

```python
nemo_gym.core.model_call_sampling.model_call_sampling_config(
    global_config_dict: typing.Optional[collections.abc.Mapping[str, typing.Any]]
) -> typing.Optional[nemo_gym.core.model_call_sampling.ModelCallSamplingConfig]
```

Return the run's `model_call_sampling` block, or `None` when the run sets none.

Raises `ValueError` when `required_model_servers` names a server that is not a Model Server of the run: a
misspelled name would otherwise leave the trained agent's Model Server unenforced, without an error.

```python
nemo_gym.core.model_call_sampling.model_sampling_context(
    segment: typing.Optional[str]
) -> collections.abc.Iterator[None]
```

Give the model calls made inside the block the segment value `segment`.

```python
nemo_gym.core.model_call_sampling.model_sampling_from_run_body(
    body: pydantic.BaseModel | collections.abc.Mapping[str, typing.Any] | None
) -> typing.Optional[nemo_gym.core.model_call_sampling.ModelCallSampling]
```

Return the values a run request or row carries under `_ng_model_sampling`, or `None`.

```python
nemo_gym.core.model_call_sampling.model_sampling_path_segment(
    segment: typing.Optional[str]
) -> str
```

Return `/ng-sampling/<segment>`, or `""` for no segment.

```python
nemo_gym.core.model_call_sampling.with_model_sampling_segment(
    url_path: str,
    segment: typing.Optional[str]
) -> str
```

Insert `/ng-sampling/<segment>` into a Model Server generation path, where its correlation prefix puts it.

Other paths, and paths that already carry a segment or a malformed prefix, are returned unchanged.

```python
nemo_gym.core.model_call_sampling.MODEL_SAMPLING_HEADER = 'x-ng-sampling'
```

```python
nemo_gym.core.model_call_sampling.MODEL_SAMPLING_NULL = 'null'
```

```python
nemo_gym.core.model_call_sampling._ENCODED = re.compile('[A-Za-z0-9_-]+')
```

```python
nemo_gym.core.model_call_sampling._GENERATION_API_PATHS = frozenset({'/v1/chat/completions', '/v1/responses', '/v1/messages'})
```

```python
nemo_gym.core.model_call_sampling._MAX_ENCODED_LENGTH = 1024
```

```python
nemo_gym.core.model_call_sampling._MODEL_SAMPLING_SEGMENT: ContextVar[Optional[str]] = ContextVar('nemo_gym_model_sampling_segment', default=None)
```