Skip to navigation

nemo_automodel.components.models.deepseek_v41.pipeline

View as MarkdownOpen in Claude

Model-owned pipeline partition dependencies and CSA2 tensor wire layouts.

Module Contents

Classes

NameDescription
DeepseekV41BackboneOutputBackbone values before or after the stage-local final norm.
DeepseekV41PipelineMixinPreserve model dependencies across the configured contiguous stages.

API

class nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41BackboneOutput(
hidden_states: torch.Tensor,
pre_mix: torch.Tensor,
residual_streams: tuple[torch.Tensor, ...] | None,
)
Dataclass

Backbone values before or after the stage-local final norm.

attention_state
DeepseekV41AttentionState

Immutable CSA2 values described by DeepseekV41AttentionState.

hidden_states
Tensor

[batch, local_sequence, streams, hidden] on an intermediate stage, or normalized [batch, local_sequence, hidden] on the last stage.

pre_mix
Tensor

FP32 coefficients [batch, local_sequence, streams]. They remain differentiable across a pipeline boundary.

residual_streams
tuple[Tensor, ...] | None

Optional pre-block states, each [batch, local_sequence, streams, hidden], for parity diagnostics.

class nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin()

Preserve model dependencies across the configured contiguous stages.

config
DeepseekV41Config
lm_head
Module | None
model
Module
nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin._pack_pipeline_state(
boundary: int
) -> tuple[torch.Tensor, ...]

Expose immutable live CSA2 tensors to native PP autograd.

Parameters:

state
DeepseekV41AttentionState

Values with layouts in DeepseekV41AttentionState. Compressed KV keeps its gradient history; frozen index keys use a byte view so PP does not allocate or expect meaningless floating gradients.

boundary
int

Index of the next language layer.

Returns: torch.Tensor

Empty tuple at a dead-state boundary. Otherwise: differentiable KV

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin._pipeline_layer_bounds() -> tuple[int, int]

Validate a nonempty contiguous stage and return its half-open layer range.

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin._shared_state_metas(
boundary: int,
batch: int,
sequence: int,
dtype: torch.dtype
) -> tuple[torch.Tensor, ...]

Describe live state tensors using the layouts of _pack_pipeline_state.

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin._shared_state_ratio(
boundary: int
) -> int

Return the live KV ratio at a supplied cut, or zero if its state is dead.

This determines the wire contract, not the partition. Window-only layers preserve a lineage; a new Full layer replaces it before any consumer.

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin._unpack_pipeline_state(
tensors: tuple[torch.Tensor, ...],
boundary: int,
hidden_states: torch.Tensor,
packed: bool

Restore stage-local immutable CSA2 state without detaching shared KV.

Parameters:

tensors
tuple[torch.Tensor, ...]

Wire tensors documented by _pack_pipeline_state.

boundary
int

Index of this stage’s first language layer.

hidden_states
torch.Tensor

Tensor [batch, local_sequence, streams, hidden] whose device and compute dtype also define the incoming KV contract.

packed
bool

Whether token metadata contains packed document IDs.

Returns: DeepseekV41AttentionState

CSA2 state with layouts in DeepseekV41AttentionState. Its tensors

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin.customize_pipeline_stage_modules(
module_names_per_stage: list[list[str]],
layers_prefix: str,
text_model: torch.nn.Module | None = None
) -> list[list[str]]

Retain the supplied layer split and attach stage-owned dependencies.

Manual module_fqns_per_model_part uses the framework’s existing complete-module-list contract. This hook augments framework-generated splits; it does not choose or move layer boundaries.

nemo_automodel.components.models.deepseek_v41.pipeline.DeepseekV41PipelineMixin.get_pipeline_stage_metas(
is_first: bool,
microbatch_size: int,
seq_len: int,
dtype: torch.dtype
) -> tuple[tuple[torch.Tensor, ...], tuple[torch.Tensor, ...]]

Describe local CP-shard tensors sent between pipeline stages.

Returns: tuple[torch.Tensor, ...]

Input/output meta tensors. The intermediate payload consists of