core.models.multimodal.llava_spec

Module Contents

Functions

decoder_model_with_transformer_engine_default_spec	LLava decoder TE spec (uses Transformer Engine components).
decoder_model_with_local_default_spec	LLava decoder local spec.

API

core.models.multimodal.llava_spec.decoder_model_with_transformer_engine_default_spec(

num_experts: Optional[int] = None,

moe_grouped_gemm: bool = False,

qk_layernorm: bool = False,

) → megatron.core.transformer.spec_utils.ModuleSpec

LLava decoder TE spec (uses Transformer Engine components).

core.models.multimodal.llava_spec.decoder_model_with_local_default_spec(

num_experts: Optional[int] = None,

moe_grouped_gemm: bool = False,

qk_layernorm: bool = False,

) → megatron.core.transformer.spec_utils.ModuleSpec

LLava decoder local spec.