Skip to navigation

nemo_automodel.components.models.lfm3.parallelization

View as MarkdownOpen in Claude

Activation-checkpointing policy of LFM3.

Module Contents

Classes

NameDescription
Lfm3ModelParallelizerCheckpoint whole decoder layers, which recomputes the convolution layers in backward as well.

Data

PARALLELIZER

API

class nemo_automodel.components.models.lfm3.parallelization.Lfm3ModelParallelizer()

Bases: ModelParallelizer

Checkpoint whole decoder layers, which recomputes the convolution layers in backward as well.

nemo_automodel.components.models.lfm3.parallelization.Lfm3ModelParallelizer._use_full_layer_activation_checkpointing(
model: torch.nn.Module
) -> bool

A decoder layer takes all its inputs as arguments and the model builds no cache in training.

nemo_automodel.components.models.lfm3.parallelization.PARALLELIZER = Lfm3ModelParallelizer()