NVFP4 block-scale conversion
cudnn.ops.nvfp4_block_scale_quantize and
cudnn.ops.nvfp4_block_scale_dequantize are explicit conversion operations for
the NVFP4 tensor representation consumed by block-scaled GEMMs. They are
experimental frontend-only APIs and require CuTeDSL >= 4.7.0.
The conversion is intentionally separate from GEMM. Quantized activations can be reused by multiple projections, and callers can see and manage the packed data and scale-factor lifetime directly. No graph-topology matcher or hidden two-kernel executor is involved.
Tensor contract
The initial implementation accepts contiguous PyTorch CUDA tensors on SM100 or newer:
M must be positive and divisible by 128. K must be positive and divisible
by 64, with at most 1,024 sixteen-element groups per row. The packed bytes are
an implementation-compatible NVFP4 representation, not a promise that a
different quantizer will make identical midpoint choices.
Direct FROST composition requires a PyTorch build that exposes
torch.float4_e2m1fn_x2, because FROST’s TVM-FFI signature is typed as FP4.
On older PyTorch builds the standalone converter instead returns the same bytes
in a torch.uint8 carrier; that carrier remains valid for explicit storage and
the standalone dequantizer, but is not accepted directly by the current FROST
GEMM launch boundary.
High-level use
The high-level quantizer allocates its two outputs and returns a TupleDict, so
both tuple unpacking and named access (result["packed_tensor"],
result["scale_tensor"]) are supported. The dequantizer returns its BF16
tensor directly.
Explicit plan lifecycle
Nvfp4BlockScaleQuantizer and Nvfp4BlockScaleDequantizer accept sample input
and preallocated output tensors. Call check_support(), compile(), then
execute(...). The compiled K is fixed, while M is symbolic and may change
between executions as long as it remains divisible by 128. execute() never
allocates, converts, or copies its tensor arguments.
Both kernels in this package were authored for cuDNN Frontend from the public
NVFP4 arithmetic and F8_128x4 layout contract; they do not embed code copied
from another framework’s quantizer.
Requires CuTeDSL >= 4.7.0. Older installed versions are rejected before the kernel module is imported; the package-wide dependency floor is 4.8.0.