> This page is for version Nightly (default).
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.7.0 · 26.10: https://docs.nvidia.com/nemo/automodel/v0.7/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.loss.decision

Post-training objectives for decision models (per-question distributions over a finite option set).

Every question of every record is one *decision*: the model emits one logit per allowed option and the answer
distribution is a softmax over them.  Questions have different numbers of options, so a batch is represented as a
padded `[questions, options]` logit matrix plus a boolean option mask, where `questions` counts every question of
every record in the batch and `options` is the largest option count among them.

The configured decision objective supports two stages:

* **SFT** - label-smoothed cross-entropy on the gold option, plus a Brier term (calibration) and a ranked-probability
  score (squared EMD) for ordinal `score` questions. The optional ranked-probability term applies only to
  ordinal questions.

* **RL** - the decision is a one-step contextual bandit, so the policy gradient can be computed two ways:

  * `expected_reward`: maximise `E_{a~pi}[r(a)] = sum_a pi(a) r(a)` exactly.  With a finite option set this is the
    exact policy gradient without option-sampling variance `sum_a pi(a) (r(a) - E[r]) grad log pi(a)`.
  * `grpo`: sample `num_samples` options per question, use the group-mean (optionally std-normalised) reward as
    the baseline and apply a grouped REINFORCE surrogate for single-step decisions.

  Both add a KL penalty to a frozen reference policy (normally the SFT model) and an optional proper-scoring anchor
  (log score of the gold option). These objectives do not guarantee calibration.

The same module owns the tutorial-specific group-masked bidirectional InfoNCE
objective and experimental Gaussian-proposal RLCR continuation.

## Module Contents

### Classes

| Name                                                                                | Description                                                             |
| ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`DecisionLoss`](#nemo_automodel-components-loss-decision-DecisionLoss)             | Decision objective; construct it with `DecisionLossConfig.build`.       |
| [`DecisionLossConfig`](#nemo_automodel-components-loss-decision-DecisionLossConfig) | Hyper-parameters of the decision objectives (see the module docstring). |
| [`RLCRLossOutput`](#nemo_automodel-components-loss-decision-RLCRLossOutput)         | Scalar loss terms from calibration-aware policy continuation.           |

### Functions

| Name                                                                                                                | Description                                                               |
| ------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [`_masked_mean`](#nemo_automodel-components-loss-decision-_masked_mean)                                             | Reduce eligible per-question values without inventing a zero-weight loss. |
| [`group_masked_bidirectional_infonce`](#nemo_automodel-components-loss-decision-group_masked_bidirectional_infonce) | Compute the CLM bidirectional in-batch InfoNCE objective.                 |
| [`masked_log_softmax`](#nemo_automodel-components-loss-decision-masked_log_softmax)                                 | Log-softmax over valid options.                                           |
| [`rlcr_loss`](#nemo_automodel-components-loss-decision-rlcr_loss)                                                   | Apply JevAny-style RLCR to closed-set decision logits.                    |

### Data

[`IGNORE_INDEX`](#nemo_automodel-components-loss-decision-IGNORE_INDEX)

[`SCORE_TYPE`](#nemo_automodel-components-loss-decision-SCORE_TYPE)

### API

```python
class nemo_automodel.components.loss.decision.DecisionLoss(
    config: nemo_automodel.components.loss.decision.DecisionLossConfig
)
```

**Bases:** `Module`

Decision objective; construct it with `DecisionLossConfig.build`.

```python
nemo_automodel.components.loss.decision.DecisionLoss._effective_weights(
    targets: nemo_automodel.components.loss.decision_batch.DecisionTargets
) -> torch.Tensor
```

Return per-question objective weights.

**Parameters:**

**`targets`** `DecisionTargets`

Supervision tensors with layouts documented by DecisionTargets.

---

**Returns:** `torch.Tensor`

Tensor of shape \[questions], zero where required feedback is unavailable.

```python
nemo_automodel.components.loss.decision.DecisionLoss._rewards(
    targets: nemo_automodel.components.loss.decision_batch.DecisionTargets,
    onehot: torch.Tensor
) -> torch.Tensor
```

staticmethod

Per-option rewards.

**Parameters:**

**`targets`** `DecisionTargets`

Supervision tensors.

---

**`onehot`** `torch.Tensor`

Float tensor of shape \[questions, options] with the gold option one-hot (fallback reward).

---

**Returns:** `torch.Tensor`

Float tensor of shape \[questions, options].

```python
nemo_automodel.components.loss.decision.DecisionLoss.forward(
    logits: torch.Tensor,
    targets: nemo_automodel.components.loss.decision_batch.DecisionTargets,
    generator: torch.Generator | None = None
) -> tuple[torch.Tensor, dict[str, torch.Tensor]]
```

Compute the configured objective.

**Parameters:**

**`logits`** `torch.Tensor`

Tensor of shape \[questions, options]; padding entries (`targets.option_mask` False) are ignored.

---

**`targets`** `DecisionTargets`

Supervision tensors (see `DecisionTargets`) on the same device as `logits`.

---

**`generator`** `torch.Generator | None` — default: None

Optional RNG for GRPO sampling.

---

**Returns:** `torch.Tensor`

`(loss, metrics)`: `loss` is a scalar weighted mean over questions; `metrics` maps names to detached

```python
nemo_automodel.components.loss.decision.DecisionLoss.normalization_weight(
    targets: nemo_automodel.components.loss.decision_batch.DecisionTargets
) -> torch.Tensor
```

Return the denominator for composing microbatch or data-parallel losses.

**Parameters:**

**`targets`** `DecisionTargets`

Supervision tensors with layouts documented by DecisionTargets.

---

**Returns:** `torch.Tensor`

Scalar tensor containing the sum of eligible per-question weights. A

```python
class nemo_automodel.components.loss.decision.DecisionLossConfig(
    mode: typing.Literal['sft', 'rl'] = 'sft',
    label_smoothing: float = 0.05,
    brier_weight: float = 1.0,
    ordinal_weight: float = 0.5,
    rl_objective: typing.Literal['expected_reward', 'grpo'] = 'expected_reward',
    num_samples: int = 8,
    sample_temperature: float = 1.0,
    normalize_advantages: bool = True,
    kl_coef: float = 0.05,
    anchor_weight: float = 0.1,
    question_weights: dict[str, float] = dict()
)
```

Dataclass

Hyper-parameters of the decision objectives (see the module docstring).

**`anchor_weight`** `float = 0.1`

Weight of the supervised gold-option log-score anchor during RL.

---

**`brier_weight`** `float = 1.0`

SFT weight of the Brier score.

---

**`kl_coef`** `float = 0.05`

Weight of `KL(pi || pi_ref)` (needs reference log-probabilities).

---

**`label_smoothing`** `float = 0.05`

SFT label smoothing over the valid options of each question.

---

**`mode`** `Literal['sft', 'rl'] = 'sft'`

`"sft"` for supervised fine-tuning or `"rl"` for reward-driven post-training.

---

**`normalize_advantages`** `bool = True`

Divide GRPO advantages by the group reward std.

---

**`num_samples`** `int = 8`

GRPO samples per question.

---

**`ordinal_weight`** `float = 0.5`

SFT weight of the ranked probability score on `score` questions.

---

**`question_weights`** `dict[str, float] = field(default_factory=dict)`

Optional `{question_id: weight}`.

---

**`rl_objective`** `Literal['expected_reward', 'grpo'] = 'expected_reward'`

`"expected_reward"` (exact policy gradient) or `"grpo"` (sampled, group baseline).

---

**`sample_temperature`** `float = 1.0`

GRPO sampling temperature.

---

```python
nemo_automodel.components.loss.decision.DecisionLossConfig.build() -> 'DecisionLoss'
```

Return the loss module for this configuration.

```python
class nemo_automodel.components.loss.decision.RLCRLossOutput(
    loss: torch.Tensor,
    policy_loss: torch.Tensor,
    supervised_loss: torch.Tensor,
    mean_reward: torch.Tensor,
    brier: torch.Tensor
)
```

Dataclass

Scalar loss terms from calibration-aware policy continuation.

**`brier`** `Tensor`

---

**`loss`** `Tensor`

---

**`mean_reward`** `Tensor`

---

**`policy_loss`** `Tensor`

---

**`supervised_loss`** `Tensor`

---

```python
nemo_automodel.components.loss.decision._masked_mean(
    values: torch.Tensor,
    weights: torch.Tensor
) -> torch.Tensor
```

Reduce eligible per-question values without inventing a zero-weight loss.

**Parameters:**

**`values`** `torch.Tensor`

Tensor of shape \[questions].

---

**`weights`** `torch.Tensor`

Tensor of shape \[questions], zero where supervision is absent.

---

**Returns:** `torch.Tensor`

Scalar tensor; a zero-weight batch stays connected to the input graph.

```python
nemo_automodel.components.loss.decision.group_masked_bidirectional_infonce(
    state_projections: torch.Tensor,
    action_projections: torch.Tensor,
    logit_scale: torch.Tensor,
    group_ids: torch.Tensor | None = None
) -> torch.Tensor
```

Compute the CLM bidirectional in-batch InfoNCE objective.

**Parameters:**

**`state_projections`** `torch.Tensor`

Tensor of shape \[batch, projection].

---

**`action_projections`** `torch.Tensor`

Tensor of shape \[batch, projection] with row-aligned positives.

---

**`logit_scale`** `torch.Tensor`

Scalar Tensor storing the log inverse temperature.

---

**`group_ids`** `torch.Tensor | None` — default: None

Optional Tensor of shape \[batch]. Off-diagonal rows with the same ID are false negatives and masked.

---

**Returns:** `torch.Tensor`

Scalar Tensor containing the mean of state-to-action and action-to-state cross-entropy.

```python
nemo_automodel.components.loss.decision.masked_log_softmax(
    logits: torch.Tensor,
    mask: torch.Tensor
) -> torch.Tensor
```

Log-softmax over valid options.

**Parameters:**

**`logits`** `torch.Tensor`

Tensor of shape \[questions, options]; padding entries may hold any value (including `-inf`).

---

**`mask`** `torch.Tensor`

Bool tensor of shape \[questions, options]; True marks a real option.

---

**Returns:** `torch.Tensor`

Float32 tensor of shape \[questions, options].  Padding entries are exactly 0 (not `-inf`) so that products

```python
nemo_automodel.components.loss.decision.rlcr_loss(
    logits: torch.Tensor,
    targets: torch.Tensor,
    group_size: int,
    exploration_std: float,
    policy_weight: float,
    supervised_weight: float,
    generator: torch.Generator | None = None
) -> nemo_automodel.components.loss.decision.RLCRLossOutput
```

Apply JevAny-style RLCR to closed-set decision logits.

This is an experimental adaptation, not an upstream CLEF or CLM recipe.
It samples Gaussian logit proposals, uses correctness minus squared
confidence error as reward, subtracts a per-example group baseline, and
retains a supervised cross-entropy anchor.

**Parameters:**

**`logits`** `torch.Tensor`

Tensor of shape \[batch, options].

---

**`targets`** `torch.Tensor`

Tensor of shape \[batch] with zero-based correct option indices.

---

**`group_size`** `int`

Number of perturbed proposals sampled for each decision.

---

**`exploration_std`** `float`

Positive Gaussian standard deviation in logit space.

---

**`policy_weight`** `float`

Multiplier for the score-function policy term.

---

**`supervised_weight`** `float`

Multiplier for the cross-entropy anchor.

---

**`generator`** `torch.Generator | None` — default: None

Optional random generator controlling proposal sampling.

---

**Returns:** `RLCRLossOutput`

Named scalar loss terms. Every Tensor has shape \[].

```python
nemo_automodel.components.loss.decision.IGNORE_INDEX = -100
```

```python
nemo_automodel.components.loss.decision.SCORE_TYPE = 2
```