> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/llms.txt. # Sparse Attention These pages describe attention APIs with selected tokens, blocks, or masks, and related compression operations. The APIs are experimental and subject to change. | API | Use | | --- | --- | | [Native Sparse Attention (NSA)](/operations/sparse_attention/nsa) | Selection, compression, top-k, and sliding-window attention. | | [DeepSeek Sparse Attention (DSA)](/operations/sparse_attention/dsa) | Sparse attention and its indexer, score, and top-k operations. | | [Simplified Sparse Attention](/operations/sparse_attention/simplified_sparse_attention) | Packed attention with a trainable top-k indexer. | | [Query Sparse Attention (QSA)](/operations/sparse_attention/qsa) | Attention over selected four-token blocks, with bitmap and indexed training APIs. | | [HSTU Attention](/operations/sparse_attention/hstu_attention) | Masked attention for recommender models, with SiLU scores instead of softmax. | | [HSTU LMSD](/operations/sparse_attention/hstu_lmsd) | Fused layer normalization, multiplication, SiLU, and dropout. | | [Block Sparse Attention (BSA)](/operations/sparse_attention/bsa) | Attention over a caller-supplied block-sparse mask. | | [CSA Fused Compressor](/operations/sparse_attention/csa) | Compression for compressed sparse attention, with aligned HCA backward. | | [Flex Attention](/operations/sparse_attention/flex_attention) | Attention with reusable static-mask plans. | See [Attention](/operations/Attention) for scaled dot-product attention and [Linear Attention](/operations/LinearAttention) for recurrent attention APIs. ## Docs - [Native Sparse Attention (NSA)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/nsa.md) - [Simplified Sparse Attention](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/simplified_sparse_attention.md) - [DeepSeek Sparse Attention (DSA)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/dsa.md) - [Query Sparse Attention (QSA)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/qsa.md) - [Indexed QSA training](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/qsa/qsa_indexed.md) - [HSTU Attention (Blackwell SM100/SM103)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/hstu_attention.md) - [HSTU LayerNorm-Multiply-SiLU-Dropout (LMSD)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/hstu_lmsd.md) - [Block Sparse Attention (BSA)](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/bsa.md) - [CSA Fused Compressor](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/csa.md) - [Aligned HCA Backward](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/csa/aligned_hca_backward.md) - [Flex Attention](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/flex_attention.md) - [Flex Attention Mask Plan Design](https://docs.nvidia.com/cudnn/latest/operations/sparse_attention/flex_attention/flex_attention_design.md)