nemo_gym.telemetry.endpoints
nemo_gym.telemetry.endpoints
Span wrappers for Gym’s endpoint handlers.
Applied in the three SimpleServer subclasses where routes are registered, so every one
of Gym’s ~150 servers is instrumented without touching any of them individually.
Why a wrapper and not the FastAPI auto-instrumentation alone: the instrumentor gives one
SERVER span per HTTP request, named after the route. That is the right thing for the
transport, but it cannot know that /run is a rollout and /verify is a verification, it
cannot be switched on per span group, and it cannot attach Gym’s rollout id. These
wrappers add the semantic layer on top.
The rollout id comes from nemo_gym.core.episode_context.current_rollout_id, the
ContextVar Gym already sets from RolloutContextMiddleware and the agent’s /run
wrapper. It is bridged onto the span rather than replaced: one correlation scheme, now
visible from traces, Gym’s own logs, and captured trajectories alike.
Module Contents
Functions
Data
API
The reward of a /verify result; nothing for a masked sample or a result without a numeric reward.
Wrap an async FastAPI handler in a span-group-gated span.
functools.wraps sets __wrapped__, which is what FastAPI’s inspect.signature
follows to build the request model — so the route keeps its body type, its validation
and its OpenAPI schema. Gym already relies on this for
SimpleResponsesAPIAgent.run_with_rollout_context.
Parameters:
Span group gating this site. Checked at call time, not decoration time:
span groups are configured during init_telemetry, long after import.
Span name, e.g. gym.verify.
The async handler to wrap.
Attributes constant for this route, e.g. the server name. Evaluated once at wrap time, not per request.
Maps the handler’s return value to attributes set on the span before it ends.
Returns: Callable
The wrapped handler.
traced_endpoint for the agent’s /run, plus gym.rollout.duration_ms.
One /run is one rollout, which makes this the span everything else in a rollout
hangs off — the model calls and verifications it triggers become its descendants
through W3C context propagation.
traced_endpoint for /verify, plus the gym.verify.* metrics.
succeeded records whether the verification call completed, not whether the task
passed. Reward and accuracy are experiment telemetry and belong in W&B, not in an
application-telemetry metric — see
kb/knowledge/concepts/application-vs-experiment-telemetry.md. A verifier that
correctly scores an answer as wrong is a success here; a verifier that raises is not.
The gym.verify span carries the reward as nemo.gym.rollout.reward, which a metric label could not.