> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo/automodel/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/automodel/_mcp/server.

# nemo_automodel.components.models.deepseek_v41.packing

One packed text layout for direct forwards and contiguous context parallelism.

## Module Contents

### Classes

| Name                                                                                                  | Description                                                       |
| ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| [`PackedSequenceLayout`](#nemo_automodel-components-models-deepseek_v41-packing-PackedSequenceLayout) | Compression-aligned packed rows and reversible token coordinates. |

### Functions

| Name                                                                                    | Description                                                             |
| --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [`packed_layout`](#nemo_automodel-components-models-deepseek_v41-packing-packed_layout) | Validate packed spans and align document starts for compression groups. |

### API

```python
class nemo_automodel.components.models.deepseek_v41.packing.PackedSequenceLayout(
    source_indices: torch.Tensor,
    input_positions: torch.Tensor,
    position_ids: torch.Tensor,
    sequence_ids: torch.Tensor
)
```

Dataclass

Compression-aligned packed rows and reversible token coordinates.

**`input_positions`** `Tensor`

---

**`position_ids`** `Tensor`

---

**`sequence_ids`** `Tensor`

---

**`source_indices`** `Tensor`

---

```python
nemo_automodel.components.models.deepseek_v41.packing.PackedSequenceLayout.pack(
    values: torch.Tensor,
    fill: int | float = 0
) -> torch.Tensor
```

Insert alignment padding without changing real token order.

**Parameters:**

**`values`** `torch.Tensor`

Tensor of shape \[batch, original\_sequence, ...], with arbitrary trailing axes.

---

**`fill`** `int | float` — default: 0

Value for padding slots.

---

**Returns:** `torch.Tensor`

Independent tensor \[batch, padded\_sequence, ...], preserving gradients of real tokens.

```python
nemo_automodel.components.models.deepseek_v41.packing.PackedSequenceLayout.restore(
    values: torch.Tensor
) -> torch.Tensor
```

Restore caller coordinates, returning zeros at original padding slots.

**Parameters:**

**`values`** `torch.Tensor`

Tensor of shape \[batch, padded\_sequence, ...], with arbitrary trailing axes.

---

**Returns:** `torch.Tensor`

Independent tensor \[batch, original\_sequence, ...] retaining real-token gradients.

```python
nemo_automodel.components.models.deepseek_v41.packing.packed_layout(
    seq_lens: torch.Tensor,
    seq_lens_padded: torch.Tensor | None,
    input_shape: tuple[int, int],
    alignment: int,
    minimum_length: int = 0
) -> nemo_automodel.components.models.deepseek_v41.packing.PackedSequenceLayout
```

Validate packed spans and align document starts for compression groups.

**Parameters:**

**`seq_lens`** `torch.Tensor`

Integer real document lengths \[batch, documents] or \[documents] for batch one.
Zero and -1000 entries denote absent documents.

---

**`seq_lens_padded`** `torch.Tensor | None`

Optional integer physical span lengths with the same shape. These include
existing per-document padding; omitted means documents are directly concatenated.

---

**`input_shape`** `tuple[int, int]`

Original \[batch, sequence] dimensions.

---

**`alignment`** `int`

Positive multiple for each document's physical span.

---

**`minimum_length`** `int` — default: 0

Minimum aligned row length, used to retain a fixed pack budget.

---

**Returns:** `PackedSequenceLayout`

Layout whose tensor fields are documented by PackedSequenceLayout, on seq\_lens.device.