> This page is for version Latest (default).
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/cudnn/latest/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/cudnn/llms.txt.

# Indexed QSA training

These attention calls **consume existing indices**; they do not compute the
indexer score or top-k. See QSA entry points and the indexer ([docs](/cudnn/operations/sparse_attention/qsa#finding-qsa-attention-and-its-indexer), [GitHub](https://github.com/NVIDIA/cudnn-frontend/blob/develop/docs/operations/sparse_attention/qsa.md#finding-qsa-attention-and-its-indexer))
for block selection, bitmap training and the backward engine.

`cudnn.ops.qsa_indexed_forward` and `qsa_indexed_backward` accept packed token
indices directly. The implementation lives in `cudnn.QSA.indexed` and shares
QSA's attention kernels. It supports BF16
12Q/1KV and 6Q/1KV (one KV head shared by twelve or six query heads) with head
dimension 256 on SM100/SM103 with CuTeDSL >= 4.7 and on SM107 with
CuTeDSL >= 4.8 (the backward needs the CUDA 13 build of the DSL).

```python
from cudnn.ops import qsa_indexed_forward, qsa_indexed_backward

out, out_fp32, lse = qsa_indexed_forward(q, k, v, indices, scale=1 / 16)
dq, dk, dv = qsa_indexed_backward(grad_out, q, k, v, out_fp32, lse, indices, scale=1 / 16)
```

These explicit calls return `TupleDict` objects in the order shown. They do not
register an autograd function. Save the original FP32 output and natural-log
LSE; casting the BF16 output back to FP32 is insufficient. Packed documents
reach the backward through `cu_seqlens` (int32 `[B + 1]` prefix sums) and
`max_seq_len` with `selection_mode="causal_topk"` (below). Without `cu_seqlens`
the rows are one sequence, which stays exact for packed documents but serves
only the first document on the dense prefix path.

| Tensor | Shape | Dtype |
|---|---|---|
| Q, output, grad_out, dQ | `[T, Hq, 256]` | BF16 |
| K, V, dK, dV | `[T, 1, 256]` | BF16 |
| indices | `[T, W]` | int32 |
| out_fp32 | `[T, Hq, 256]` | FP32 |
| lse | `[Hq, T]` | FP32 |

`Hq` is 12 or 6. All tensors are contiguous on one CUDA device. Data tensors
require 16-byte alignment; indices and LSE require four-byte alignment. `T > 0`
and `T * Hq * 256 <= INT32_MAX`. Scale is a finite host scalar.

`W` is a positive multiple of four, at most 2052. Each four-slot group describes
one distinct complete block: `[start, start+1, start+2, start+3]`, or four `-1`
padding slots. A partial causal tail is a consecutive prefix followed by `-1`.
Valid tokens are absolute packed token IDs, belong to the query's document,
and never exceed the query's absolute token ID. A tail begins at the document's
own block origin; packed document starts need not be divisible by four. Groups
may appear in any order and padding groups may occur between selected groups.
Groups must not overlap or repeat within a row. These device-value relationships
are caller contracts; execution does not read indices back to the host. A row's
blocks accumulate in the order of its groups, so the result bytes follow that
order.

The Relax layout with 512 complete four-token groups plus a four-slot causal
tail fits directly, including invalid tail slots. A 2051-wide carrier must be
padded explicitly by the caller; this API does not silently convert it. Empty
rows produce zero output/gradients and negative-infinity LSE. Selection is not
differentiated. Pass the same selection and scale to forward and backward.

A GPU hash table compacts the selected starts. Backward runs the engine
described in QSA ([docs](/cudnn/operations/sparse_attention/qsa), [GitHub](https://github.com/NVIDIA/cudnn-frontend/blob/develop/docs/operations/sparse_attention/qsa.md))
over them. A nonfinite K/V value of a masked token adds exact zeros, and one of
a selected token poisons its queries. Every call rebuilds metadata, including
during CUDA Graph replay. The backward compiles different kernels below and
above the 22528-token dense crossover, from 196608 tokens on, and for each
`deterministic` value. Before capture, warm up with the query head count,
token counts, `max_seq_len` and `deterministic` the capture will use.

The public wrappers allocate outputs and one workspace. The internal APIBase
plan validates current storage and uses only caller-owned buffers in execute;
its cache retains immutable geometry and compiled kernels. Indexed metadata
storage is proportional to `T * W`, independent of the full key-block count.
At 128K with W=2052, compact IDs occupy 256.5 MiB in either direction, compared
with 16 GiB for the bitmap entry's `[T, NB]` IDs. The 4 GiB bitmap is also
unnecessary. These are metadata sizes, not total process
memory; saved state, outputs and FP32 gradient accumulators are additional.

## Complete blocks and causal tails

Packed documents and the top-512 selection contract are declared through the
same public entry point:

```python
# cu_seqlens is a CUDA int32 tensor, for example [0, 43691, 87382, 131072].
out, out_fp32, lse = qsa_indexed_forward(q, k, v, indices)
dq, dk, dv = qsa_indexed_backward(
    grad_out, q, k, v, out_fp32, lse, indices,
    selection_mode="causal_topk", cu_seqlens=cu_seqlens,
    max_seq_len=43691,
)
```

`selection_mode="general"` remains the default. The `causal_topk` mode declares
the following stricter input contract; it is never inferred from tensor shapes:

- Q/K/V, grad_out and saved FP32 output are finite. LSE is finite and comes
  from forward with the same indices and scale.
- `W=2052` and `scale=1/16`. For query position `p` within its document, select
  exactly `min((p + 1) // 4, 512)` distinct, complete four-token blocks from
  that document's causal prefix, followed by its incomplete causal tail.
  Complete blocks fill consecutive groups starting at column zero, in any
  order. No padding holes or empty selections are allowed. Before 512 blocks
  are available, pad the unused groups with `-1`; the tail may follow the
  complete blocks or occupy the last four slots, as in Relax. After 512 blocks
  are available, the tail occupies the last four slots. Invalid tail slots
  and all unused slots are `-1`.
- Packed inputs require `cu_seqlens`: contiguous device int32 prefix sums
  `[B + 1]`, starting at zero, ending at `T`, and nondecreasing. Empty documents
  and document starts not divisible by four are supported. Indices use
  absolute packed token IDs; block origins restart at each document boundary.
  Without `cu_seqlens`, all `T` tokens are one document.
- Optional `max_seq_len` is a host integer in `[1, T]` bounding every document
  length. It defaults to `T`. A bound below the 22528-token dense crossover
  plans no sparse path, so the workspace is smaller and no sparse launches run.
  Every document of every replay must respect this bound. The kernel
  rereads device `cu_seqlens` on replay; boundaries are not cached.

These device-value promises are caller contracts, not host-side checks.
Use `general` for arbitrary selections. Passing `cu_seqlens` or `max_seq_len`
with `general` is rejected because that mode obtains its bounds from indices.
The keyword-only arguments preserve existing positional calls.
`deterministic` (`False` by default) applies to both modes.
`deterministic=True` repeats the bytes when every
document's groups lie on one four-token grid, which `causal_topk` requires and
`general` does not check. Both modes run the
backward engine described in QSA ([docs](/cudnn/operations/sparse_attention/qsa), [GitHub](https://github.com/NVIDIA/cudnn-frontend/blob/develop/docs/operations/sparse_attention/qsa.md)),
which also describes `deterministic` and gives the record
and workspace sizing.

The 12Q/1KV and 6Q/1KV backward requires the **CUDA 13 CuTeDSL compiler** in
addition to the CuTeDSL versions and architectures above. A CUDA 13 PyTorch
build alone does not establish the DSL compiler version; unsupported compiler
builds are rejected before kernel compilation.

The indexed entry is useful when a producer already has token IDs. Use the unchanged Relax
comparison in `benchmark/qsa/benchmark_relax.py` for the intended workload;
Miles ratios do not establish a speedup over Relax, and attention microbenchmarks
do not measure selector, TE, tensor-parallel communication or model throughput.