`core.models.huggingface.qwen_model`#

Module Contents#

Classes#

QwenHuggingFaceModel

Wrapper for Qwen LM HuggingFace models.

API#

class core.models.huggingface.qwen_model.QwenHuggingFaceModel(config)#

Bases: megatron.core.models.huggingface.HuggingFaceModule

Wrapper for Qwen LM HuggingFace models.

Initialization

_fsdp_modules#: None

forward(*args, **kwargs)#: Qwen forward.

embedding(input_ids, position_ids=None)#: Function to run process tokens with input embeddings