core.transformer.enums

Module Contents

Classes

ModelType	Model Type
LayerType	Layer type embedding: embedding layer loss: loss layer encoder: encoder layer, not implemented yet, expect to be used in MLLM models decoder: decoder layer mtp: multi-token prediction layer, not implemented yet
AttnType	Attention type
AttnMaskType	Attention Mask Type
AttnBackend	Attention Backend

API

class core.transformer.enums.ModelType(*args, **kwds)

Bases: enum.Enum

Model Type

encoder_or_decoder for bert, gpt etc encoder_and_decoder for multimodal , T5 etc

Initialization

encoder_or_decoder

1

property encoder_and_decoder

Deprecated property - use encoder_or_decoder instead.

class core.transformer.enums.LayerType(*args, **kwds)

Bases: enum.Enum

Layer type embedding: embedding layer loss: loss layer encoder: encoder layer, not implemented yet, expect to be used in MLLM models decoder: decoder layer mtp: multi-token prediction layer, not implemented yet

Initialization

embedding

1

loss

2

encoder

3

decoder

4

mtp

5

class core.transformer.enums.AttnType(*args, **kwds)

Bases: enum.Enum

Attention type

Initialization

self_attn

1

cross_attn

2

class core.transformer.enums.AttnMaskType(*args, **kwds)

Bases: enum.Enum

Attention Mask Type

Initialization

padding

1

causal

2

no_mask

3

padding_causal

4

arbitrary

5

causal_bottom_right

6

class core.transformer.enums.AttnBackend(*args, **kwds)

Bases: enum.Enum

Attention Backend

Initialization

flash

1

fused

2

unfused

3

local

4

auto

5