nemo_automodel.components.models.kimi_k3.kda_fused.chunk_kda_bwd_triton
nemo_automodel.components.models.kimi_k3.kda_fused.chunk_kda_bwd_triton
Chunked Kimi Delta Attention BACKWARD in Triton for the Kimi-K3 call. From the raw
forward inputs q, k, v, g, beta, cu_seqlens and the output gradient do it recomputes the forward intermediates and
produces dq, dk, dv (bf16) and dg, dbeta (fp32) — nothing is saved from the forward. Measured on GB200 against FLA
0.4.2’s autograd backward (headline mode with saved intermediates) at the Kimi-K3 layer-microbatch shape: 4.25-4.39 ms
vs 4.89-4.95 ms, every output within per-output tolerances (dq/dk/dv/dg atol 2e-2 rtol 2e-2, dbeta atol 2e-1 rtol 5e-2, matched
ratio >= 0.99), identical numerics on document layouts it was not tuned on. Entry point: run.
Module Contents
Functions
Data
API
(I + X)^-1 for a strictly-lower-triangular 32x32 X, via the doubling identity (I+X)^-1 = (I-X)(I+X^2)(I+X^4)(I+X^8)(I+X^16).
FOUR squarings, not five. After s steps the running product equals sum_{j=0}^{2^(s+1)-1} (-X)^j; a 32x32 strictly-lower X has X^32 = 0, so s=4 (terms 0..31) is EXACT and a 5th step would multiply by I+X^32 = I. The 64x64 version needs terms 0..63, hence the 5 steps in the current code. Verified: s=4 -> err 1.6e-14 vs exact, s=3 -> 5.8e-5 (insufficient).
Backward of the K3 chunk_kda call (destination-passing): fills dq, dk, dv (bf16) and dg, dbeta (fp32).
q, k: [1, T, H, 128] bf16; v: [1, T, H, 128] bf16; g: [1, T, H, 128] fp32 log gate in (-5, 0); beta: [1, T, H] fp32;
cu_seqlens: [N + 1] int32 document offsets; do: [1, T, H, 128] bf16 gradient of the output. Every forward
intermediate is recomputed here. The per-chunk metadata needs the document offsets on the host: pass them as
cu_seqlens_cpu (a sequence of ints, e.g. derived from a dense batch’s shape) to avoid the device->host sync
that cu_seqlens.tolist() otherwise costs — the same contract as FLA’s own cu_seqlens_cpu.