Skip to navigation
DevelopmentAPI ReferenceFull Library ReferenceNemo AutomodelNemo AutomodelComponentsModelsMinimax M3 VlKernelsnemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100

nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100

View as Markdown

Delta preprocess for the MiniMax M3 MSA SM100 backward.

Computes delta[t, h] = sum_d out[t, h, d] * grad_out[t, h, d] in FP32 from BF16 inputs, which the backward subtracts from dP to form dS. One CTA of 256 threads reduces a [128, 128] tile of one head; rows past T are predicated off.

Adapted from FlashAttention’s CuTe DSL backward preprocess (flash_attn/cute/flash_bwd_preprocess.py, Copyright (c) 2025, Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, Tri Dao; BSD-3-Clause), specialized to the fixed [T, 64, 128] MSA layout.

Module Contents

Classes

NameDescription
_MSABackwardPreprocessSm100Row-wise sum(O * dO) over head_dim for the THD contract.

Functions

NameDescription
_compileCompile the delta reduction once per process with a dynamic token count.
run_preprocessEnqueue delta[t, h] = sum_d out[t, h, d] * grad_out[t, h, d].

Data

COPY_BITS

COPY_ELEMS

NUM_THREADS

THREADS_PER_ROW

TILE_M

API

class nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100._MSABackwardPreprocessSm100()

Row-wise sum(O * dO) over head_dim for the THD contract.

nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100._MSABackwardPreprocessSm100.__call__(
mO: cutlass.cute.Tensor,
mdO: cutlass.cute.Tensor,
mDelta: cutlass.cute.Tensor,
stream: cuda.bindings.driver.CUstream
)
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100._MSABackwardPreprocessSm100.kernel(
mO: cutlass.cute.Tensor,
mdO: cutlass.cute.Tensor,
mDelta: cutlass.cute.Tensor,
tiled_copy: cutlass.cute.TiledCopy
)
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100._compile() -> typing.Any

Compile the delta reduction once per process with a dynamic token count.

nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.run_preprocess(
out: torch.Tensor,
grad_out: torch.Tensor,
delta: torch.Tensor
) -> None

Enqueue delta[t, h] = sum_d out[t, h, d] * grad_out[t, h, d].

Parameters:

out
torch.Tensor

Contiguous BF16 [T, 64, 128] forward output.

grad_out
torch.Tensor

Contiguous BF16 [T, 64, 128] output gradient, on the same device.

delta
torch.Tensor

Contiguous FP32 [T, 64] written in place.

nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.COPY_BITS = 128
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.COPY_ELEMS = COPY_BITS // cutlass.BFloat16.width
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.NUM_THREADS = 256
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.THREADS_PER_ROW = HEAD_DIM // COPY_ELEMS
nemo_automodel.components.models.minimax_m3_vl.kernels.msa_backward_preprocess_sm100.TILE_M = 128