> This page is for version Latest (default).
> For other versions, use one of these documentation indexes:
> - Latest (default): https://docs.nvidia.com/cudnn/latest/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/cudnn/llms.txt.

# Causal Conv1d Decode Update

**This FE-OSS API is experimental and subject to change.**

`cudnn.ops.causal_conv1d_update` advances a mutable depthwise
causal-convolution cache in place and returns the output for one decode step.
The semantic tensor contract uses `weight[D, W]` and `conv_state[S, D, L]`,
where `L >= W - 1`. The first native implementation targets the width-four
short convolution used by GDN/KDA-style linear-attention blocks. Compilation,
architecture dispatch, output ownership, streams, and kernel schedules remain
private implementation details.

Use [`cudnn.ops.causal_conv1d`](/cudnn/operations/CausalConv1d/causal_conv1d) for full-sequence prefill or
training. Its width-four final state can be passed directly to this operation.

Without circular-buffer metadata, row `n`, channel `d`, and selected cache
slot `s` are updated as follows:

```text
history = concat(conv_state[s, d, :], x[n, d])
conv_state[s, d, :] = history[-L:]
acc[n, d] = sum_j history[-W + j] * weight[d, j] + bias[d]
output[n, d] = activation(acc[n, d])
```

The convolution, bias addition, and optional SiLU use FP32. The output is
converted to the input dtype. The bias is zero when omitted.
`activation=None` and `"identity"` select the identity epilogue.
`"silu"` and `"swish"` select the same fused SiLU specialization.
Identity and SiLU compile as separate kernels.

When `conv_state_indices` is omitted, row `n` selects slot `n`. When it is
present, `s = conv_state_indices[n]`. A value of `-1` denotes a padding row:
its output is zero and it does not mutate `conv_state`. Other selected slots
must be in range and unique within the decode batch because `conv_state` is
mutated; repeated padding rows are allowed.

`cache_seqlens` is a reserved compatibility keyword. The current operation
accepts only `None` and raises `NotImplementedError` otherwise. A future
implementation may use it for circular-buffer state without changing the
public signature.

## Python API

```python
import torch
import cudnn

x = torch.randn(8, 2048, device="cuda", dtype=torch.bfloat16)
weight = torch.randn(2048, 4, device="cuda", dtype=torch.bfloat16)
conv_state = torch.randn(8, 2048, 4, device="cuda", dtype=torch.bfloat16)

output = cudnn.ops.causal_conv1d_update(
    x,
    conv_state,
    weight,
    bias=None,
    activation="silu",
)
```

The signature is:

```python
cudnn.ops.causal_conv1d_update(
    x,
    conv_state,
    weight,
    bias=None,
    activation=None,
    *,
    cache_seqlens=None,
    conv_state_indices=None,
) -> torch.Tensor
```

`conv_state` is mutated in place and the return value is an ordinary newly
allocated Tensor, not a wrapper result. The operation registers the mutation
with `torch.library`, including its FakeTensor contract. It is inference-only;
autograd inputs are rejected.

The fourth and fifth positional arguments remain `bias` and `activation`.
State-routing metadata is keyword-only. The first call for a supported device,
dtype, shape, optional-input signature, and activation performs JIT compilation.
Warm the exact signature before latency measurement or CUDA Graph capture.

## Semantic tensor contract

- `x`: FP16 or BF16 `[N, D]` for one decode token, with strides `(ld, 1)`; compact
  `ld == D` accepts every `D`, while padded `ld > D` requires `ld % 8 == 0`
  so every row starts at a 16-byte-aligned address
- `weight`: `[D, W]` with the dtype of `x`
- `conv_state`: `[S, D, L]` with the dtype of `x`, updated in place, with `L >= W - 1`; compact
  storage is accepted, and an `L=3` state returned by
  `cudnn.ops.causal_conv1d` is accepted without copying
- `bias`: optional `[D]` with the dtype of `x`
- `cache_seqlens`: reserved compatibility keyword; currently must be `None`
- `conv_state_indices`: optional int32 `[N]` state-slot selection; `-1` is padding
- output: `[N, D]` with the dtype of `x`

A future multi-token extension can admit `x[N, D, Tstep]` without changing
the state or weight meanings. The current public implementation rejects 3D
`x` rather than silently interpreting its layout.

## Current native implementation

- GPU: compute capabilities 8.0, 8.6, 8.7, 8.9, 9.0, 10.0, 10.3, 11.0,
  12.0, and 12.1; the portable one-row schedule is used on every admitted
  target
- performance-characterized GPU: B200 SM100
- `x`: FP16 or BF16 `[N, D]` with strides `(ld, 1)`; compact `ld == D` accepts every
  channel count, and padded `ld > D` requires `ld % 8 == 0`
- `weight`: contiguous `[D, 4]` with the dtype of `x`
- `conv_state`: `[S, D, L]` with the dtype of `x` and `L` equal to 3 or 4, updated in place;
  compact storage is accepted for both lengths, and `L=3` additionally accepts
  the channel-last `(3 * D, 1, D)` stride returned by full-sequence prefill
- `bias`: optional contiguous `[D]` with the dtype of `x`
- `cache_seqlens`: must be `None` or omitted
- `conv_state_indices`: optional contiguous CUDA int32 `[N]`
- output: contiguous `[N, D]` with the dtype of `x`
- activation: identity or SiLU
- autograd: unsupported
- pointer alignment: 16 bytes for FP16 and BF16 tensors and 4 bytes for indices

The native kernel requires CUTLASS DSL 4.7 or newer for the inline-PTX
integration it imports. The package-wide `cutedsl` extra keeps its broader
`>=4.5` floor for unrelated APIs; installing only that minimum is not
sufficient for this operation. The optional FLA 0.5.2 adapter treats the
resulting `ImportError` as a typed decline and executes FLA's original path.

BF16 runtime correctness was validated on A100 SM80, L40S SM89, H200 SM90, B200
SM100, an SM103 board, and RTX 5080 SM120. SM86, SM87, SM110,
and SM121 have compile-only validation. The SM110 path is functional support,
not a training-performance claim. FP16 runtime correctness and deterministic
state updates were validated on GeForce RTX 4090 SM89.

For width four, `L=3` is the standard `W - 1` final state handed off by
prefill; it uses a functionally correct stride-aware scalar state-access
specialization and therefore needs no layout conversion.
`L=4` retains the original vectorized fast path used by GDN/KDA decode. Other
semantically valid widths and state lengths, circular-buffer updates,
multi-token updates, speculative intermediate-state returns, prefill, and
training currently raise a clear unsupported-configuration error. The indexed
path is functional paged-state support; its duplicate-index validation is not
performance-characterized as a fast path. For the current native path, the
kernel checks indices on device and traps on values below `-1`, out-of-range
slots, or duplicate non-padding slots;
the resulting CUDA error is asynchronous and the failed update is not
transactional.

## Semantic provenance and benchmarking

Behavioral references are FLA 0.5.2 `ShortConvolution.step` (MIT) and the
public `causal_conv1d_update` contract from Dao-AILab/causal-conv1d at revision
`cd81f0413cad2fc1e6f17e785ac39f59aae690cd` (BSD-3-Clause). No source from
either project is included. The implementation uses CUTLASS/CuTe DSL, inline
PTX, and in-tree NVIDIA FROST primitives.

Use `benchmark/causal_conv1d_update_sm100.py` for route-proof, correctness, and
an interleaved comparison against FLA. It intentionally uses the private
preallocated plan so kernel timing does not include public output allocation
or custom-op dispatch. The benchmark records the actual GPU architecture and
software environment; Slurm metadata is optional.