Trajectory Capability Matrix

Use this reference to check which evidence an agent path provides. Start with Rollout Evidence for collection and artifact relationships.

✅ Full means complete for the documented path; 🟡 Partial means some fields or paths are missing; ❌ Unavailable means the path does not supply that evidence. These are manually maintained coverage claims, not guarantees for every configuration.

Model Calls and History

Capture requires observability_enabled, a shared model_call_capture_dir, and a rollout-prefixed Gym Model Server URL. Direct-provider routes bypass capture. Token counts depend on provider usage fields. History may come from captured payloads or harness conversations.

Agent Activity

Turns require explicit semantic-turn records. Tool execution covers output, status, and timestamps; parallel timing requires independent tool intervals. Resource/sandbox coverage means that the benchmark path emits standardized agent evidence, not that every resource metric is available.

The other agents listed above report these four dimensions as unavailable. Model HTTP capture alone does not establish tool execution or sandbox activity.

Path Qualifications

Inspect ng_trajectory.gaps before relying on partial coverage. Aggregate-metrics requests omit the trajectory and source attachments.

ATIF export

gym eval export --format atif converts the supported ng_trajectory version 1.0 subset into ATIF v1.7 without loading NeMo Relay. The initial converter accepts a completed root invocation with one initial system/user input phase, exactly one captured model call per turn, text-only output (including ordered multipart tool results), and unambiguous function-call and function-result relationships. Model calls may be joined by model_call_id or by a unique (model_ref, response_id) pair. It rejects later user/system turns and source records when their status, evidence gaps, content type, invocation structure, tool correlation, or provider completion evidence would require dropping or guessing information. This strict boundary means an accepted conversion is complete for the supported source profile; it does not claim that the original agent harness observed every action outside Gym.

Responses message phase values such as commentary and final_answer are rejected because ATIF v1.7 has no equivalent field. Namespaced Responses function calls are also rejected because the standard ATIF tool-call fields cannot retain the separate namespace identity. A missing or null phase or namespace is accepted because it carries no additional source meaning. Provider-native hosted tools, shell and patch calls, tool search, compaction records, approvals, and additional-tool records remain outside the initial generic function-call profile and are rejected rather than flattened.

The semantic profile maps ordered messages, reasoning, function calls and results, turn identity and resolution, tool timing, normalized model identity, and normalized token counts. Training token IDs, log probabilities, and routed-expert metadata are outside the initial profile and fail conversion rather than being copied into an extension. Captured provider request and response values are retained opaquely under step-level extra.nemo_gym.model_call metadata because ATIF’s standard step fields do not carry those provider-specific payloads. The converter does not parse those raw payloads or use them to reconstruct the canonical trajectory; the producer-owned response_metadata and token_stats fields supply completion, model, and usage evidence. Per-step token fields are preserved when available. Final prompt, completion, and cached-token totals are emitted only when every model call provides the corresponding field, so unavailable values are never counted as zero. Each agent step retains the normalized model identity. The root agent model is populated only when every model-backed step uses the same known model; routed or fallback trajectories keep their model identities on the individual steps instead.

The strict ATIF reverify path additionally requires complete prompt and completion counts when any supported usage is present because one Gym response cannot represent partial standard usage. A partial-metric export remains valid ATIF for other consumers, but Gym reverify rejects it instead of dropping the known counts or reporting a false complete aggregate.

The ATIF agent.name comes from each rollout row’s agent_ref.name. The initial batch exporter accepts one agent identity per input file because the caller supplies one corresponding agent implementation version. Conversion provenance is kept under the ATIF root’s extra.nemo_gym namespace rather than overloading the agent identity fields:

The initial strict converter does not emit partial files or conversion warnings. Unsupported or lossy inputs fail validation. A future best-effort mode would need an explicit partial status and structured warnings rather than reusing complete.

Terminus2 decisions and invocation ownership

With model-call capture enabled as described above, terminus_2_sandboxed_agent emits an AgentInvocation for each identified rollout execution. Its execution-specific x-session-id header lets the collector link captured model calls to that invocation, including retries without a response ID and summarization calls.

The adapter records TrajectoryTurn entries at Harbor’s main-agent decision boundary, before terminal commands execute. Each observed decision retains the exact input sent for its selected response, the answer and reasoning, and an exact model-server/response-ID reference. Its timestamp is the observed response time. turn_no counts main decision attempts; step_count counts completed nonempty terminal-command batch attempts before that decision. A returned batch timeout counts as an attempt, not proof that every command ran. One decision can request several commands. resolved remains unset because a model’s completion claim does not establish task success.

Decisions remain available after parser errors, tool failures, or cancellation. Responses rejected by the adapter and synthetic Harbor fallback messages are not presented as accepted main-agent decisions. Their available model-call evidence remains invocation-owned, with diagnostic gaps. Known summarization boundaries also emit ContextCompactionObservation records with exact response references; auxiliary fallback calls are not mislabeled as main decisions. Missing or reused response IDs produce gaps instead of guessed associations. turns_unavailable is reported only when no actual decision response could be represented.

Observed turns enable rollout_missing_agent_turns and agent_turn_hollow. Invocation ownership enables model_call_failed, model_call_missing_token_counts, model_call_zero_completion_tokens, model_call_runaway_generation, and trajectory_capture_mismatch under their documented capture and payload requirements. See Rollout health checks.

The underlying HTTP transport can retry after server admission without notifying the adapter. Complete turn-call accounting therefore remains unavailable: transport_retry_visibility_unavailable and turn_model_call_scope_incomplete keep rollout token reconciliation and full policy-call coverage unobserved while preserving those seven checks. This producer does not establish an overall healthy rollout or task success from calls.