`nemo_automodel._transformers.v4_patches.layer_types`#

Widen transformers’ ALLOWED_LAYER_TYPES so legacy custom configs load.

Some community models (e.g. nvidia/Nemotron-Flash-1B) ship a custom configuration_*.py whose layer_types entries (e.g. 'deltanet', 'f', 'm2', 'a') are not in the upstream allow-list. Loading them via AutoConfig.from_pretrained triggers validate_layer_type and raises StrictDataclassClassValidationError before model instantiation.

The validator performs a module-global lookup of ALLOWED_LAYER_TYPES at call time, so rebinding it in place takes effect on subsequent validations.

Module Contents#

Classes#

_LayerTypesPatchFinder

Meta-path finder that applies patch_allowed_layer_types the moment transformers.configuration_utils finishes loading.

Functions#

`patch_allowed_layer_types`	Extend `transformers.configuration_utils.ALLOWED_LAYER_TYPES` in place.
`install_layer_types_patch_hook`	Ensure `patch_allowed_layer_types` runs before any call to `PreTrainedConfig.validate_layer_type`.
`_noop_validate_layer_type`	No-op replacement for `PretrainedConfig.validate_layer_type`.
`relax_layer_types_validator`	Disable `PretrainedConfig.validate_layer_type` and its registered copies.

Data#

`logger`
`_TARGET_MODULE`
`DEFAULT_EXTRA_LAYER_TYPES`
`_PATCHED`
`_HOOK_INSTALLED`
`_VALIDATOR_RELAXED`
`__all__`

API#

nemo_automodel._transformers.v4_patches.layer_types.logger#: ‘getLogger(…)’

nemo_automodel._transformers.v4_patches.layer_types._TARGET_MODULE#: ‘transformers.configuration_utils’

nemo_automodel._transformers.v4_patches.layer_types.DEFAULT_EXTRA_LAYER_TYPES: tuple[str, ...]#: (‘deltanet’, ‘f’, ‘m2’, ‘a’)

nemo_automodel._transformers.v4_patches.layer_types._PATCHED: bool#: False

nemo_automodel._transformers.v4_patches.layer_types.patch_allowed_layer_types( extra: Iterable[str] = DEFAULT_EXTRA_LAYER_TYPES, ) → bool[source]#

Extend transformers.configuration_utils.ALLOWED_LAYER_TYPES in place.

Idempotent and best-effort: any failure (missing attribute, transformers not installed, unexpected container type) is logged and swallowed so the caller’s import path is not broken.

Parameters:: extra – Layer-type names to append if not already present.
Returns:: True if the tuple was modified on this call, False otherwise.

nemo_automodel._transformers.v4_patches.layer_types._HOOK_INSTALLED: bool#: False

class nemo_automodel._transformers.v4_patches.layer_types._LayerTypesPatchFinder[source]#

Bases: importlib.abc.MetaPathFinder

Meta-path finder that applies patch_allowed_layer_types the moment transformers.configuration_utils finishes loading.

The finder only intercepts the single target module name. For that name it delegates to the remaining finders to obtain a real spec, then wraps the loader’s exec_module so the patch runs once, post-exec. All deviations from the happy path (missing loader, unexpected spec shape, patch failure) are swallowed with a log — the finder must never break imports.

Initialization

find_spec(fullname, path=None, target=None)[source]#

nemo_automodel._transformers.v4_patches.layer_types.install_layer_types_patch_hook() → bool[source]#

Ensure patch_allowed_layer_types runs before any call to PreTrainedConfig.validate_layer_type.

Two paths:

If transformers.configuration_utils is already loaded, patch immediately.
Otherwise, register a meta-path finder that patches on first import.

Idempotent; safe to call multiple times.

Returns:: True if a hook was installed (or the patch was applied directly on this call), False if a previous call already arranged one.

nemo_automodel._transformers.v4_patches.layer_types._VALIDATOR_RELAXED: bool#: False

nemo_automodel._transformers.v4_patches.layer_types._noop_validate_layer_type(self)[source]#: No-op replacement for PretrainedConfig.validate_layer_type.

nemo_automodel._transformers.v4_patches.layer_types.relax_layer_types_validator() → bool[source]#

Disable PretrainedConfig.validate_layer_type and its registered copies.

Some upstream configs (e.g. stepfun-ai/Step-3.5-Flash) ship layer_types whose length differs from num_hidden_layers; newer transformers rejects such configs, and huggingface_hub wraps that ValueError in StrictDataclassClassValidationError. Suppressing the validator lets downstream code load the config unmodified.

The huggingface_hub @strict decorator freezes validator references on each class into __class_validators__ at decoration time, so swapping PretrainedConfig.validate_layer_type alone is insufficient: every already-decorated subclass keeps its own list. This helper rewrites both the class attribute and each list entry across the live subclass tree, then newly-decorated subclasses pick up the no-op automatically via getattr(cls, "validate_layer_type") at their decoration time.

Idempotent and best-effort: failures (missing attribute, transformers not installed) are logged and swallowed so the caller is never broken.

Returns:: True if the patch was applied on this call, False otherwise.

nemo_automodel._transformers.v4_patches.layer_types.__all__#: [‘DEFAULT_EXTRA_LAYER_TYPES’, ‘install_layer_types_patch_hook’, ‘patch_allowed_layer_types’, ‘relax_…

nemo_automodel._transformers.v4_patches.layer_types#