`core.packed_seq_params`#

Module Contents#

Classes#

PackedSeqParams

parameters to TEDotProductAttention and fused rope kernels for the thd (packed) sequence format

API#

class core.packed_seq_params.PackedSeqParams#

parameters to TEDotProductAttention and fused rope kernels for the thd (packed) sequence format

qkv_format: str#: None

cu_seqlens_q: torch.Tensor#: None

cu_seqlens_kv: torch.Tensor#: None

cu_seqlens_q_padded: torch.Tensor#: None

cu_seqlens_kv_padded: torch.Tensor#: None

max_seqlen_q: int#: None

max_seqlen_kv: int#: None