nemo_automodel.components.speculative.eagle.vllm_target

vLLM adapter for the EAGLE-3 target backend.

A thin engine adapter on top of the engine-agnostic contract in nemo_automodel.components.speculative.eagle.target_runner: it builds a vLLM runner and wraps it in RunnerEagle3TargetModel. It mirrors nemo_automodel.components.speculative.eagle.sglang_target exactly, only swapping the engine, so the supervision contract (shift / aux-concatenation semantics, aux-layer defaulting, embedding access) is shared and a vLLM run is numerically equivalent to the co-located HuggingFace one.

vLLM is imported lazily (only inside VLLMEagle3TargetModel.from_pretrained), and the vLLM-internal forward is isolated further in nemo_automodel.components.speculative.eagle.vllm_runner, so importing this module never pulls in vLLM.

Module Contents

Classes

Name	Description
`VLLMEagle3TargetModel`	EAGLE-3 target backend whose runner is vLLM.

API

class nemo_automodel.components.speculative.eagle.vllm_target.VLLMEagle3TargetModel()

Bases: RunnerEagle3TargetModel

EAGLE-3 target backend whose runner is vLLM.

Adds only vLLM construction; the supervision contract is inherited from RunnerEagle3TargetModel.

nemo_automodel.components.speculative.eagle.vllm_target.VLLMEagle3TargetModel.from_pretrained(
    model_path: str,
    aux_layer_ids: typing.Sequence[int] | None = None,
    dtype: torch.dtype | None = None,
    tp_size: int = 1,
    trust_remote_code: bool = False,
    vllm_kwargs = {}
) -> 'VLLMEagle3TargetModel'

classmethod

Build a vLLM runner for model_path and wrap it as a target backend.

vLLM is imported here (not at module load) so this module stays importable in environments without vLLM; vllm_kwargs are passed through to vllm.LLM for endpoint / parallelism / memory tuning.