bridge.peft.module_matcher#

Module Contents#

Classes#

ModuleMatcher

Implements the LoRA (Low-Rank Adaptation) module for parameter-efficient fine-tuning.

Data#

logger

HAVE_TE

API#

bridge.peft.module_matcher.logger#

‘getLogger(…)’

bridge.peft.module_matcher.HAVE_TE#

‘all(…)’

class bridge.peft.module_matcher.ModuleMatcher#

Implements the LoRA (Low-Rank Adaptation) module for parameter-efficient fine-tuning.

LoRA uses a low-rank projection to adapt the weights of a pre-trained model to a new downstream task. This class facilitates the application of LoRA to specific modules within the model architecture.

Parameters:

target_modules (List[str], optional) – A list of module names to apply LoRA to. Defaults to all linear layers [‘linear_qkv’, ‘linear_proj’, ‘linear_fc1’, ‘linear_fc2’]. - ‘linear_qkv’: Apply LoRA to the fused linear layer used for query, key, and value projections in self-attention. - ‘linear_proj’: Apply LoRA to the linear layer used for projecting the output of self-attention. - ‘linear_fc1’: Apply LoRA to the first fully-connected layer in MLP. - ‘linear_fc2’: Apply LoRA to the second fully-connected layer in MLP. Target modules can also contain wildcards. For example, you can specify target_modules=[’.layers.0..linear_qkv’, ‘.layers.1..linear_qkv’] to add LoRA to only linear_qkv on the first two layers.

target_modules: List[str]#

‘field(…)’

exclude_modules: List[str]#

‘field(…)’

canonical_mapping: Dict[str, Set]#

‘field(…)’

_pattern_to_alias: Dict[str, Set[str]]#

‘field(…)’

_alias_to_pattern: Dict[str, str]#

‘field(…)’

_alias_matches: Dict[str, Set[str]]#

‘field(…)’

_init_target_match_state() → None#

(Re)initialize target-module alias bookkeeping from the current target_modules.

Called from PEFT.__call__ immediately before the validation walk so that any post-construction mutation of self.target_modules is reflected. Subclasses that derive aliases from target_modules differently (e.g. CanonicalLoRA) override this method.

match(
m: torch.nn.Module,
name: Optional[str] = None,
prefix: Optional[str] = None,
) → Optional[tuple[str, str]]#

Determines whether a given module matches specified target patterns.

This function checks if the provided module m should be included based on predefined mapping rules (canonical_mapping, target_modules, and exclude_modules). It returns the matching pattern if a match is found; otherwise, it returns None.

Parameters:

  • m (nn.Module) – The module being checked.

  • name (str, optional) – The module’s name.

  • prefix (str, optional) – A prefix to be used in constructing full_name.

Returns:

A tuple containing (matching_pattern, full_name) if a match is found; otherwise, None.

Return type:

Optional[Tuple[str, str]]

Matching Logic:

  1. If canonical_mapping is defined, it checks:

  • Whether name exactly matches a pattern.

  • Whether full_name matches any regex pattern in canonical_mapping.

  1. If target_modules is defined, it follows the same logic as canonical_mapping.

  2. If neither canonical_mapping nor target_modules are defined, it ensures:

  • name is not in exclude_modules.

  • full_name does not match any target_modules patterns.

  • m is an instance of nn.Linear.

Notes:

  • exclude_modules should only be non-empty if neither canonical_mapping nor target_modules are set.

  • The function asserts that exclude_modules is empty when using canonical_mapping or target_modules.

register_target_alias(alias: str, pattern: str) → None#

Associate a user-supplied alias with the canonical pattern used for matching.

For plain LoRA an alias is identical to its pattern. For CanonicalLoRA an alias such as "linear_q" maps to the canonical pattern "linear_qkv".

_record_match(pattern: str, full_name: Optional[str]) → None#

Track which aliases successfully matched modules during traversal.

_reset_target_match_state() → None#

Clear per-call match tracking without discarding the alias registrations.

_validate_target_matches() → None#

Warn if any requested target aliases failed to match a module.

Intended to be called after a full validation walk over the model. Skips when no aliases are registered (e.g. exclude_modules mode). A warning (not an error) keeps the previous default behavior — recipes whose target_modules defaults list more entries than a given model exposes (e.g. CanonicalLoRA defaults vs. a model without fused linear_qkv) continue to work without per-recipe overrides.