nemo_automodel.components.speculative.eagle.registry#
Dispatch registry mapping target architecture -> EAGLE draft model.
Mirrors SpecForge’s modeling/auto.py pattern but keyed by HF
architectures string instead of by config class. The string key avoids a
hard import dependency on every individual HF *Config class (which can
differ between transformers versions) and matches NeMo AutoModel’s existing
_transformers/registry.MODEL_ARCH_MAPPING style.
Today the dense draft (LlamaEagle3DraftModel / LlamaEagleDraftModel)
covers all registered architectures: the implementation is config-driven and
reads attention_bias, mlp_bias, head_dim, rope_theta /
rope_scaling, and rms_norm_eps directly from the target config.
Adding an architecture that fits this shape is a one-line registry
append. Architectures that need a different draft (e.g. gpt-oss with its
alternating sliding-window attention and non-standard RoPE) get a new
draft_cls entry pointing at a dedicated draft module.
Module Contents#
Classes#
How to build an EAGLE draft model for a particular target architecture. |
Functions#
Return the first registered draft spec matching any architecture in the list. |
|
Resolve the EAGLE-3 draft spec for a target’s |
|
Resolve the EAGLE-1 / EAGLE-2 draft spec for a target’s |
Data#
API#
- class nemo_automodel.components.speculative.eagle.registry.DraftSpec#
How to build an EAGLE draft model for a particular target architecture.
- draft_cls: type[transformers.PreTrainedModel]#
None
- nemo_automodel.components.speculative.eagle.registry._DENSE_ARCHITECTURES: tuple[str, ...]#
(‘LlamaForCausalLM’, ‘Phi3ForCausalLM’, ‘Qwen3ForCausalLM’, ‘Qwen3MoeForCausalLM’)
- nemo_automodel.components.speculative.eagle.registry.EAGLE3_DRAFT_REGISTRY: dict[str, nemo_automodel.components.speculative.eagle.registry.DraftSpec]#
None
- nemo_automodel.components.speculative.eagle.registry.EAGLE1_DRAFT_REGISTRY: dict[str, nemo_automodel.components.speculative.eagle.registry.DraftSpec]#
None
- nemo_automodel.components.speculative.eagle.registry._resolve(
- architectures: list[str],
- registry: dict[str, nemo_automodel.components.speculative.eagle.registry.DraftSpec],
- recipe_name: str,
Return the first registered draft spec matching any architecture in the list.
- nemo_automodel.components.speculative.eagle.registry.resolve_eagle3_draft_spec(
- architectures: list[str],
Resolve the EAGLE-3 draft spec for a target’s
config.architecturesfield.
- nemo_automodel.components.speculative.eagle.registry.resolve_eagle1_draft_spec(
- architectures: list[str],
Resolve the EAGLE-1 / EAGLE-2 draft spec for a target’s
config.architecturesfield.