Convolutions
Convolution Fprop
Convolution fprop computes:
C++ API
Conv_fprop_attributes is a lightweight structure with setters:
Python API
- conv_fprop
- image
- weight
- pre_padding
- post_padding
- stride
- dilation
- convolution_mode
- compute_data_type
- name
The symmetric-padding overload accepts padding in place of pre_padding and post_padding.
CuTe DSL Fused Conv3D + Post-Operations
The experimental direct Python APIs Conv3dRmsNormSiluSm100 and
Conv3dRmsNormSiluPadSm100 fuse Conv3D with bias, optional residual addition,
channel RMS normalization, affine scale, and SiLU. Related variants provide
raw Conv3D, bias/residual/spatial padding, and causal input packing plus Conv3D.
These APIs are not selected automatically by cudnn.pygraph.
Current Limitations
- SM100 or SM103 GPUs,
nvidia-cutlass-dsl >= 4.9, and BF16 inference only; no backward implementation. - Fixed
3x3x3convolution, unit stride and dilation, and one group. The valid-convolution APIs consume already-padded input when padding is needed. - Selected channel pairs from the WAN VAE
base_dim=160family (160/320/640), not arbitrary channels or the 128/256/512 family. Integrated normalization supports only160->160,160->320, and320->320; C640 output uses raw convolution followed by standalone normalization. The causal input variant currently supports only12->160. Unsupported configurations have no fallback. - Contiguous NTHWC activations, except
CausalConv3dWithCacheSm100, which accepts strided NCTHW input. Weights require variant-specific packing before execution. - Normalization uses an L2-norm clamp followed by
sqrt(C)scaling, with fixed BF16 rounding points; it is not arbitrary-epsilon RMSNorm. - Padding/cache layouts are fixed for streaming convolution. The fused normalization-and-padding variant writes current frames and zeros but leaves existing-history interiors for the caller to fill.
See the fused Conv3D + post-operations API reference for the complete per-variant channel table, tensor shapes, normalization math, weight formats, and history/stream contracts.
Convolution Dgrad
Convolution dgrad computes data gradient during backpropagation.
C++ API
Conv_dgrad_attributes is a lightweight structure with setters:
Python API
- conv_dgrad
- loss
- filter
- pre_padding
- post_padding
- stride
- dilation
- convolution_mode
- compute_data_type
- name
conv_dgrad infers the dims of the returned dX from loss, filter and the padding/stride/dilation
attributes; per spatial axis the inferred extent is
(S_loss - 1) * stride + (R - 1) * dilation + 1 - (pre_padding + post_padding) (minus 2 * padding for
the symmetric-padding overload). With stride > 1 the forward extent division loses the remainder, so
the inferred dX can be smaller than the forward input even without padding: a 32x32 input with a 3x3
filter infers a 31x31 dX at stride 2 and padding 0, while stride 1 with padding 1 infers 32x32. Set the
dims and strides explicitly to get a dX shaped like the forward input:
The symmetric-padding overload accepts padding in place of pre_padding and post_padding.
Convolution Wgrad
Convolution wgrad computes weight gradient during backpropagation.
C++ API
Conv_wgrad_attributes is a lightweight structure with setters:
Python API
- conv_wgrad
- image
- loss
- pre_padding
- post_padding
- stride
- dilation
- convolution_mode
- compute_data_type
- name
conv_wgrad cannot infer the dims of the returned dW from the graph inputs; set them explicitly
(the filter dims/strides) before marking it an output, otherwise graph construction fails with dims not set:
The symmetric-padding overload accepts padding in place of pre_padding and post_padding.