> This page is for version 0.5.0 · 26.06.
> For other versions, use one of these documentation indexes:
> - Nightly (default): https://docs.nvidia.com/nemo/automodel/nightly/llms.txt
> - Latest: https://docs.nvidia.com/nemo/automodel/latest/llms.txt
> - 0.5.0 · 26.06: https://docs.nvidia.com/nemo/automodel/v0.5/llms.txt
> - 0.4.0 · 26.04: https://docs.nvidia.com/nemo/automodel/v0.4/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.loss.masked_ce

## Module Contents

### Classes

| Name                                                                                 | Description                                                         |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------- |
| [`MaskedCrossEntropy`](#nemo_automodel-components-loss-masked_ce-MaskedCrossEntropy) | Cross-entropy loss that handles ignored or masked target positions. |

### API

```python
class nemo_automodel.components.loss.masked_ce.MaskedCrossEntropy(
    fp32_upcast: bool = True,
    ignore_index: int = -100,
    reduction: str = 'sum'
)
```

**Bases:** `Module`

Cross-entropy loss that handles ignored or masked target positions.

```python
nemo_automodel.components.loss.masked_ce.MaskedCrossEntropy.forward(
    logits: torch.Tensor,
    labels: torch.Tensor,
    mask: torch.Tensor | None = None,
    num_label_tokens: int | None = None,
    loss_weights: torch.Tensor | None = None
) -> torch.Tensor
```

Compute the masked cross-entropy loss between logits and targets.

If a mask is provided, the loss is computed per element, multiplied by the mask,
and then averaged. If no mask is provided, the standard cross-entropy loss is used.

**Parameters:**

**`logits`** `torch.Tensor`

The predicted logits with shape \[batch\_size, seq\_len, vocab\_size] where C is the number of classes.

---

**`labels`** `torch.Tensor`

The ground truth class indices with shape \[batch\_size, seq\_len].

---

**`mask`** `torch.Tensor` — default: None

A tensor that masks the loss computation. Items marked with
1 will be used to calculate loss, otherwise ignored. Must be broadcastable to the shape
of the loss. Defaults to None.

---

**`num_label_tokens`** `int | None` — default: None

Optional global supervised-token count used to
normalize a sum-reduced loss.

---

**`loss_weights`** `torch.Tensor | None` — default: None

Optional per-token multipliers with the same shape as
`labels`. Only supported with `reduction="sum"`.

---

**Returns:** `torch.Tensor`

torch.Tensor: The computed loss as a scalar tensor.