nemo_automodel.components.speculative.streaming

Tensor-free control-plane contracts for the speculative-training stream.

The streaming pipeline (per the EAGLE-3 / DFlash / DSpark train-inference disaggregation RFC, issue #3062 PR 1) splits every transferred sample into a :mod:refs (control-plane, no tensors) and a :mod:store (data-plane, holds the supervision tensors). The queue carries refs only; tensors live in the store and are referenced by :class:nemo_automodel.components.speculative.streaming.refs.SampleRef.feature_keys.

This package owns:

:class:~nemo_automodel.components.speculative.streaming.refs.SampleRef and :class:~nemo_automodel.components.speculative.streaming.refs.FeatureSpec — the frozen, tensor-free reference carried on every control-plane hop.
:func:~nemo_automodel.components.speculative.streaming.refs.assert_no_tensors — the guard that enforces the no-tensor invariant on the control plane.
:class:~nemo_automodel.components.speculative.streaming.store.FeatureStore — the pluggable data-plane transport (local dict, shared POSIX mount, NCCL, …); :class:~nemo_automodel.components.speculative.streaming.stores.local.LocalFeatureStore — the in-process implementation land-tested by PR 1.
:class:~nemo_automodel.components.speculative.streaming.queue.SampleRefQueue — the metadata-only lease/ack/fail queue between producers and consumers with visibility-timeout reclaim and watermark-based backpressure.
:class:~nemo_automodel.components.speculative.streaming.producer.FeatureProducer and :class:~nemo_automodel.components.speculative.streaming.loader.FeatureDataLoader — the EAGLE-3 produce and consume sides that turn a target forward into refs and refs back into Eagle3TargetBatch instances.

Subpackages

nemo_automodel.components.speculative.streaming.stores

Submodules

Package Contents

Data

__all__

API

nemo_automodel.components.speculative.streaming.__all__ = ['FeatureAlgorithm', 'FeatureDataLoader', 'FeatureProducer', 'FeatureSpec', 'Fea...