> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.loss.chunked_ce

## Module Contents

### Classes

| Name                                                                                    | Description                                       |
| --------------------------------------------------------------------------------------- | ------------------------------------------------- |
| [`ChunkedCrossEntropy`](#nemo_automodel-components-loss-chunked_ce-ChunkedCrossEntropy) | Cross-entropy loss computed over sequence chunks. |

### Functions

| Name                                                                                        | Description                                                 |
| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
| [`compute_cross_entropy`](#nemo_automodel-components-loss-chunked_ce-compute_cross_entropy) | Computes the cross-entropy loss between logits and targets. |

### Data

[`_compiled_compute_cross_entropy`](#nemo_automodel-components-loss-chunked_ce-_compiled_compute_cross_entropy)

### API

```python
class nemo_automodel.components.loss.chunked_ce.ChunkedCrossEntropy(
    chunk_len: int = 32,
    compile: bool = True,
    ignore_index: int = -100,
    reduction: str = 'sum'
)
```

**Bases:** `Module`

Cross-entropy loss computed over sequence chunks.

```python
nemo_automodel.components.loss.chunked_ce.ChunkedCrossEntropy.forward(
    logits: torch.Tensor,
    labels: torch.Tensor,
    mask: typing.Optional[torch.Tensor] = None,
    num_label_tokens: typing.Optional[int] = None
) -> torch.Tensor
```

Computes cross-entropy loss in chunks to handle long sequences more efficiently.

**Parameters:**

Model output logits of shape \[batch\_size, seq\_len, vocab\_size].

Ground-truth labels of shape \[batch\_size, seq\_len].

Boolean mask indicating valid positions (1) and
positions to ignore (0). Defaults to None.

**Returns:** `torch.Tensor`

torch.Tensor: The sum of cross-entropy losses over the sequence.

```python
nemo_automodel.components.loss.chunked_ce.compute_cross_entropy(
    logits: torch.Tensor,
    targets: torch.Tensor,
    ignore_index = -100,
    reduction = 'sum'
)
```

Computes the cross-entropy loss between logits and targets.

**Parameters:**

Model predictions of shape (sequence\_length, num\_classes).

Ground-truth labels of shape (sequence\_length,).

Target value that is ignored when computing the loss.
Defaults to -100.

**Returns:**

torch.Tensor: The sum of cross-entropy losses over the sequence.

```python
nemo_automodel.components.loss.chunked_ce._compiled_compute_cross_entropy = None
```