nemo_gym.token_id_capture
nemo_gym.token_id_capture
Provide the core training-token capture interfaces.
Training capture is separate from evaluation capture.
Middleware sets a request-scoped token sink.
The model server records a TokenEntry from its complete response.
Consumers call TokenSource.freeze for an atomic snapshot.
The snapshot includes entries and incomplete state.
Its snapshot_id identifies the exact frozen state.
TokenCaptureStore is Gym’s local sink and source implementation.
Framework transports may provide their own sink and source.
There is no HTTP token reader.
This leaf package avoids imports from Gym’s server stack.
The rollout-record finalizer needs Gym’s server stack.
It is deliberately not re-exported here.
Import nemo_gym.token_id_capture.delivery from server-side code.
The incomplete state prevents training on a rollout that lost a model call.
Finalization freezes and rebuilds the rollout.
Finalization does not retire the snapshot.
The caller retires it only after durable handoff.
Retirement uses the frozen snapshot_id and version.
Failed or masked builds retain their capture evidence.
Submodules
nemo_gym.token_id_capture.buildernemo_gym.token_id_capture.confignemo_gym.token_id_capture.consumernemo_gym.token_id_capture.deliverynemo_gym.token_id_capture.protocolsnemo_gym.token_id_capture.recordsnemo_gym.token_id_capture.sinknemo_gym.token_id_capture.store