> This page is for version 0.5.0.
> For other versions, use one of these documentation indexes:
> - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt
> - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt
> - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt
> - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt
> - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt
> - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt
> - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt
> - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server.

# nemo_gym.orchestration.executors.judge_profile

The execution profile of a judge-only job that grades a run's saved answers again.

A run that served its policy on GPUs has no allocation a judge-only job could reuse: its node pools are GPU pools and
its services are the policy's. The profile derived here keeps what grading needs from the saved submission config,
the driver with its env, mounts, container and pinned install, so the run's own judge and verifier servers start as
they did, and replaces what grading does not need. It is one of two kinds.

A run that serves nothing beside its policy gets the CPU profile: no service, no policy model, one node of a CPU
partition and a bounded walltime. The partition comes from the saved config when it declares a CPU node pool
(`gpus_per_node` 0), else from the launcher's reader placement (`NEMO_GYM_REMOTE_READER_SRUN`, the CPU partition the
heavy readers already run on), else from `--judge-partition`; without one the profile is refused, naming the three.
The run's QOS belongs to its GPU allocation and does not follow (a GPU QOS can refuse a job without GPUs): the judge
job takes `--judge-qos`, else the saved CPU pool's own QOS, else the QOS the reader placement names (`-q`/`--qos`)
for that partition, else the partition's default.

A run that serves its judge as a service of its own, on a node pool of its own beside the policy's (a vLLM judge its
judge server reaches through `GYM_POOL_*_NODES`), gets the served-judge profile: those services on their saved
node pools, with their saved shape and partition, and no policy service, policy model or policy pool. It is a GPU job
like the run, so it keeps the run's job-level QOS; `--judge-partition` and `--judge-qos` name a CPU placement and
are not used (a launcher that passes them on every recovery would put the GPU pool on a CPU partition or QOS). A
service that runs on the policy's own nodes has no served-judge profile: the judge job does not start the policy's
allocation.

Either way the walltime is the run's segment walltime bounded by the partition's MaxTime, or a default when neither is
known.

## Module Contents

### Classes

| Name                                                                                                 | Description                                                                                              |
| ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| [`JudgeExecutionProfile`](#nemo_gym-orchestration-executors-judge_profile-JudgeExecutionProfile)     | Where and for how long a judge-only job of a run grades its saved answers again, and what it starts.     |
| [`JudgeProfileUnsupported`](#nemo_gym-orchestration-executors-judge_profile-JudgeProfileUnsupported) | No judge execution profile can be derived from the submission: the message says why and what would allow |

### Functions

| Name                                                                                                 | Description                                                                                                  |
| ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [`_reader_step_option`](#nemo_gym-orchestration-executors-judge_profile-_reader_step_option)         | -                                                                                                            |
| [`_service_pools`](#nemo_gym-orchestration-executors-judge_profile-_service_pools)                   | The node pools a saved service runs on; None for a part that takes the whole allocation (no `node_pool`),    |
| [`_slurm_duration`](#nemo_gym-orchestration-executors-judge_profile-_slurm_duration)                 | -                                                                                                            |
| [`bound_judge_walltime`](#nemo_gym-orchestration-executors-judge_profile-bound_judge_walltime)       | The judge job's walltime: the run's segment walltime `segment`, bounded by the partition's MaxTime in        |
| [`choose_judge_partition`](#nemo_gym-orchestration-executors-judge_profile-choose_judge_partition)   | The CPU partition a judge-only job of this submission queues on, and where the name came from.               |
| [`is_cpu_only_submission`](#nemo_gym-orchestration-executors-judge_profile-is_cpu_only_submission)   | Whether the submission already runs on CPU nodes only: no service, no policy model, and every node pool with |
| [`judge_execution_profile`](#nemo_gym-orchestration-executors-judge_profile-judge_execution_profile) | The profile of a judge-only job of this submission on `partition` (see `choose_judge_partition`), with the   |
| [`judge_job_config`](#nemo_gym-orchestration-executors-judge_profile-judge_job_config)               | The submission config of a judge-only job of `benchmark`: the run's driver (its env, mounts, container and   |
| [`reader_step_partition`](#nemo_gym-orchestration-executors-judge_profile-reader_step_partition)     | The partition the reader step's srun arguments (`NEMO_GYM_REMOTE_READER_SRUN`) name, if any.                 |
| [`reader_step_qos`](#nemo_gym-orchestration-executors-judge_profile-reader_step_qos)                 | The QOS the reader step's srun arguments (`NEMO_GYM_REMOTE_READER_SRUN`) name, if any.                       |
| [`served_judge_placement`](#nemo_gym-orchestration-executors-judge_profile-served_judge_placement)   | The saved services beside the policy a judge-only job of this submission starts, and the saved node pools    |
| [`served_judge_profile`](#nemo_gym-orchestration-executors-judge_profile-served_judge_profile)       | The served-judge profile of a submission that serves its judge beside its policy (see                        |

### Data

[`CPU_PLACEMENT_UNUSED`](#nemo_gym-orchestration-executors-judge_profile-CPU_PLACEMENT_UNUSED)

[`CPU_PROFILE`](#nemo_gym-orchestration-executors-judge_profile-CPU_PROFILE)

[`DEFAULT_JUDGE_WALLTIME`](#nemo_gym-orchestration-executors-judge_profile-DEFAULT_JUDGE_WALLTIME)

[`JUDGE_POOL_NAME`](#nemo_gym-orchestration-executors-judge_profile-JUDGE_POOL_NAME)

[`JUDGE_WALLTIME_REASON_DEFAULT`](#nemo_gym-orchestration-executors-judge_profile-JUDGE_WALLTIME_REASON_DEFAULT)

[`PARTITION_DEFAULT_QOS`](#nemo_gym-orchestration-executors-judge_profile-PARTITION_DEFAULT_QOS)

[`SERVED_JUDGE_PROFILE`](#nemo_gym-orchestration-executors-judge_profile-SERVED_JUDGE_PROFILE)

[`_ALLOCATION_SHAPE_DIRECTIVES`](#nemo_gym-orchestration-executors-judge_profile-_ALLOCATION_SHAPE_DIRECTIVES)

[`_NO_PROFILE`](#nemo_gym-orchestration-executors-judge_profile-_NO_PROFILE)

[`_NO_SERVED_PROFILE`](#nemo_gym-orchestration-executors-judge_profile-_NO_SERVED_PROFILE)

[`_SERVED_SHAPE_DIRECTIVES`](#nemo_gym-orchestration-executors-judge_profile-_SERVED_SHAPE_DIRECTIVES)

### API

```python
class nemo_gym.orchestration.executors.judge_profile.JudgeExecutionProfile(
    partition: str,
    partition_source: str,
    walltime: str,
    walltime_bounds: tuple[str, ...],
    compute: nemo_gym.orchestration.api.SlurmComputeConfig,
    qos: str | None = None,
    qos_source: str = PARTITION_DEFAULT_QOS,
    kind: str = CPU_PROFILE,
    pools: tuple[str, ...] = (JUDGE_POOL_NAME,),
    services: tuple[str, ...] = (),
    unused: str | None = None
)
```

Dataclass

Where and for how long a judge-only job of a run grades its saved answers again, and what it starts.

**`compute`** `SlurmComputeConfig`

---

**`kind`** `str = CPU_PROFILE`

---

**`partition`** `str`

---

**`partition_source`** `str`

---

**`pools`** `tuple[str, ...] = (JUDGE_POOL_NAME,)`

---

**`qos`** `str | None = None`

---

**`qos_source`** `str = PARTITION_DEFAULT_QOS`

---

**`services`** `tuple[str, ...] = ()`

---

**`unused`** `str | None = None`

---

**`walltime`** `str`

---

**`walltime_bounds`** `tuple[str, ...]`

---

```python
nemo_gym.orchestration.executors.judge_profile.JudgeExecutionProfile.report() -> dict[str, typing.Any]
```

```python
class nemo_gym.orchestration.executors.judge_profile.JudgeProfileUnsupported()
```

**Bases:** `ValueError`

No judge execution profile can be derived from the submission: the message says why and what would allow
one.

```python
nemo_gym.orchestration.executors.judge_profile._reader_step_option(
    args: str | None,
    short: str,
    long: str
) -> str | None
```

```python
nemo_gym.orchestration.executors.judge_profile._service_pools(
    name: str,
    service: nemo_gym.orchestration.api.ServiceConfig
) -> set[str | None]
```

The node pools a saved service runs on; None for a part that takes the whole allocation (no `node_pool`),
the driver's node (a Ray cluster without `node_pools`) or the first node (a prefill/decode router).

```python
nemo_gym.orchestration.executors.judge_profile._slurm_duration(
    seconds: int
) -> str
```

```python
nemo_gym.orchestration.executors.judge_profile.bound_judge_walltime(
    segment: str | None,
    limit: dict[str, typing.Any] | None
) -> tuple[str, tuple[str, ...]]
```

The judge job's walltime: the run's segment walltime `segment`, bounded by the partition's MaxTime in
`limit` (as `_partition_time_limit` reports it), or the default when neither is known; with what bounded it.

```python
nemo_gym.orchestration.executors.judge_profile.choose_judge_partition(
    config: nemo_gym.orchestration.api.SubmitConfig,
    cluster: str,
    partition: str | None = None,
    reader_srun: str | None = None
) -> tuple[str, str]
```

The CPU partition a judge-only job of this submission queues on, and where the name came from.

`partition` (`--judge-partition`) wins; else the first saved node pool with `gpus_per_node` 0 names it; else
the reader step's arguments (`reader_srun`, the value of `NEMO_GYM_REMOTE_READER_SRUN`). Only a submission
that serves nothing beside its policy model takes the CPU profile; one that does takes the served-judge profile
(`served_judge_placement`).

```python
nemo_gym.orchestration.executors.judge_profile.is_cpu_only_submission(
    config: nemo_gym.orchestration.api.SubmitConfig
) -> bool
```

Whether the submission already runs on CPU nodes only: no service, no policy model, and every node pool with
`gpus_per_node` 0. Such a run's own compute is its judge execution profile.

```python
nemo_gym.orchestration.executors.judge_profile.judge_execution_profile(
    config: nemo_gym.orchestration.api.SubmitConfig,
    cluster: str,
    partition: str,
    partition_source: str,
    limit: dict[str, typing.Any] | None,
    qos: str | None = None,
    reader_srun: str | None = None
) -> nemo_gym.orchestration.executors.judge_profile.JudgeExecutionProfile
```

The profile of a judge-only job of this submission on `partition` (see `choose_judge_partition`), with the
partition's MaxTime `limit` bounding the walltime (see `bound_judge_walltime`).

The derived compute keeps the account, forwarded env and job-level directives of the run, less the directives
that shape the original allocation, request its GPUs or name its QOS; a saved CPU pool of the partition lends its
own directives to the judge node. The job's QOS is `qos` (`--judge-qos`), else the saved CPU pool's, else the
reader placement's (`reader_srun`, the value of `NEMO_GYM_REMOTE_READER_SRUN`) when that placement names this
partition or none, else the partition's default.

```python
nemo_gym.orchestration.executors.judge_profile.judge_job_config(
    config: nemo_gym.orchestration.api.SubmitConfig,
    cluster: str,
    benchmark: str,
    profile: nemo_gym.orchestration.executors.judge_profile.JudgeExecutionProfile,
    command: str,
    gym_ref: str | None = None
) -> nemo_gym.orchestration.api.SubmitConfig
```

The submission config of a judge-only job of `benchmark`: the run's driver (its env, mounts, container and
pinned install, at `gym_ref` when given) runs `command` in place of `gym eval run`, with no policy model, no
preparation, no resume chain, on the profile's compute. It starts the profile's services, the saved judge
services of a served-judge profile, and none of a CPU profile.

```python
nemo_gym.orchestration.executors.judge_profile.reader_step_partition(
    args: str | None
) -> str | None
```

The partition the reader step's srun arguments (`NEMO_GYM_REMOTE_READER_SRUN`) name, if any.

```python
nemo_gym.orchestration.executors.judge_profile.reader_step_qos(
    args: str | None
) -> str | None
```

The QOS the reader step's srun arguments (`NEMO_GYM_REMOTE_READER_SRUN`) name, if any.

```python
nemo_gym.orchestration.executors.judge_profile.served_judge_placement(
    config: nemo_gym.orchestration.api.SubmitConfig,
    cluster: str
) -> tuple[tuple[str, ...], dict[str, nemo_gym.orchestration.api.NodePool]] | None
```

The saved services beside the policy a judge-only job of this submission starts, and the saved node pools
they run on, in the saved order; None when it serves nothing beside its policy (the CPU profile).

A service on a node of the policy's, its pool or the whole allocation, is refused, naming it: the judge job does
not start the policy's allocation, and a separate node pool for that service would allow it.

```python
nemo_gym.orchestration.executors.judge_profile.served_judge_profile(
    config: nemo_gym.orchestration.api.SubmitConfig,
    cluster: str,
    limit: dict[str, typing.Any] | None,
    partition: str | None = None,
    qos: str | None = None
) -> nemo_gym.orchestration.executors.judge_profile.JudgeExecutionProfile
```

The served-judge profile of a submission that serves its judge beside its policy (see
`served_judge_placement`), with the kept pools' partition MaxTime `limit` bounding the walltime.

The derived compute is the run's, with only the kept node pools in their saved shape (nodes, tasks, GPUs,
partition and directives) and the bounded walltime; its account, forwarded env and job-level directives stay, less
those sizing the original allocation, which the kept pools state for themselves. It is a GPU job like the run: the
QOS stays the run's job-level one, else the kept pools', else the partition's default. `partition`
(`--judge-partition`) and `qos` (`--judge-qos`) name a CPU placement and are recorded as not used.

```python
nemo_gym.orchestration.executors.judge_profile.CPU_PLACEMENT_UNUSED = '--judge-partition/--judge-qos apply to the CPU judge profile only; not used'
```

```python
nemo_gym.orchestration.executors.judge_profile.CPU_PROFILE = 'cpu'
```

```python
nemo_gym.orchestration.executors.judge_profile.DEFAULT_JUDGE_WALLTIME = '04:00:00'
```

```python
nemo_gym.orchestration.executors.judge_profile.JUDGE_POOL_NAME = 'judge'
```

```python
nemo_gym.orchestration.executors.judge_profile.JUDGE_WALLTIME_REASON_DEFAULT = f'the default judge walltime {DEFAULT_JUDGE_WALLTIME}: the run records no segmen...
```

```python
nemo_gym.orchestration.executors.judge_profile.PARTITION_DEFAULT_QOS = "the partition's default"
```

```python
nemo_gym.orchestration.executors.judge_profile.SERVED_JUDGE_PROFILE = 'served-judge'
```

```python
nemo_gym.orchestration.executors.judge_profile._ALLOCATION_SHAPE_DIRECTIVES = frozenset({'partition', 'nodes', 'ntasks-per-node', 'gpus', 'gpus-per-node', 'gp...
```

```python
nemo_gym.orchestration.executors.judge_profile._NO_PROFILE = 'this submission has no existing CPU-only judge execution profile and none can b...
```

```python
nemo_gym.orchestration.executors.judge_profile._NO_SERVED_PROFILE = 'this submission has no served-judge execution profile'
```

```python
nemo_gym.orchestration.executors.judge_profile._SERVED_SHAPE_DIRECTIVES = _ALLOCATION_SHAPE_DIRECTIVES - {'gres', 'qos'}
```