`nemo_automodel.components._peft.module_matcher`#

Module Contents#

Classes#

ModuleMatcher

Matches Modules to apply PEFT adapters on.

Functions#

`_is_linear_module`
`_compile_wildcard_pattern`
`wildcard_match`	Return whether the pattern (target module to add LoRA) matches the key (model weight name).

Data#

API#

nemo_automodel.components._peft.module_matcher.logger#: ‘getLogger(…)’

nemo_automodel.components._peft.module_matcher._is_linear_module(module)#

nemo_automodel.components._peft.module_matcher._compile_wildcard_pattern(pattern)#

nemo_automodel.components._peft.module_matcher.wildcard_match(pattern, key)#

Return whether the pattern (target module to add LoRA) matches the key (model weight name).

Example:#

>>> wildcard_match("*.layers.0.*.linear_qkv", "decoder.layers.0.self_attention.linear_qkv")
True
>>> wildcard_match("*.layers.0.*.linear_qkv", "decoder.layers.1.self_attention.linear_qkv")
False

class nemo_automodel.components._peft.module_matcher.ModuleMatcher#

Matches Modules to apply PEFT adapters on.

Parameters:

target_modules (List[str], optional) – A list of module names to apply LoRA to. Defaults to an empty list. If empty and no other parameter is provided it will match to “_proj”. Target modules can also contain wildcards (e.g. “.layers.0..linear_qkv”). For example, you can specify target_modules=[’.layers.0..linear_qkv’, ‘.layers.1.*.linear_qkv’] to add LoRA to only linear_qkv on the first two layers.
exclude_modules (List[str], optional) – A list of module names to exclude from applying LoRA to. Defaults to an empty list. Exclude modules can also contain wildcards (e.g. “.lm_head”). For example, you can specify exclude_modules=[’.lm_head’] to exclude the lm_head.
match_all_linear (bool, optional) – Whether to match all linear layers. Defaults to False. Prefer using target_modules or exclude_modules to specify the modules to match, to avoid issues with downstream tools (e.g., vLLM, etc).
is_causal_lm (bool, optional) – Whether the model is a causal language model.

target_modules: List[str]#: ‘field(…)’

exclude_modules: List[str]#: ‘field(…)’

match_all_linear: bool#: ‘field(…)’

is_causal_lm: bool#: ‘field(…)’

__post_init__()#: Input validation.

match(m: torch.nn.Module, name: str = None, prefix: str = None)#: Return (pattern, full_name) if the module matches; otherwise None.