> This page is for version 0.7.0. > For other versions, use one of these documentation indexes: > - Main (default): https://docs.nvidia.com/nemo/gym/main/llms.txt > - 0.7.0: https://docs.nvidia.com/nemo/gym/v0.7.0/llms.txt > - 0.6.0: https://docs.nvidia.com/nemo/gym/v0.6.0/llms.txt > - 0.5.1: https://docs.nvidia.com/nemo/gym/v0.5.1/llms.txt > - 0.5.0: https://docs.nvidia.com/nemo/gym/v0.5.0/llms.txt > - 0.4.0: https://docs.nvidia.com/nemo/gym/v0.4.0/llms.txt > - 0.3.0: https://docs.nvidia.com/nemo/gym/v0.3.0/llms.txt > - 0.2.1: https://docs.nvidia.com/nemo/gym/v0.2.1/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # Trajectory Capability Matrix > Rollout-evidence coverage by agent and execution path. Use this reference to check which evidence an agent path provides. Start with [Rollout Evidence](/main/observability/rollout-evidence) for collection and artifact relationships. ✅ **Full** means complete for the documented path; 🟡 **Partial** means some fields or paths are missing; ❌ **Unavailable** means the path does not supply that evidence. These are manually maintained coverage claims, not guarantees for every configuration. > **Note** > > The statuses below retain the coverage reported in [main at 88d9d453](https://github.com/NVIDIA-NeMo/Gym/blob/88d9d45385f15b1351cdd055fff396186000f84f/fern/versions/latest/pages/reference/trajectory-capabilities.mdx). > That source does not record a full runtime-validation date or per-row test configuration. > Contributors changing an agent should update its row with the config, execution path, and validation > evidence in the same pull request. ## Model Calls and History Capture requires `observability_enabled`, a shared `model_call_capture_dir`, and a rollout-prefixed Gym Model Server URL. Direct-provider routes bypass capture. Token counts depend on provider usage fields. History may come from captured payloads or harness conversations. | Agent | Per-call metadata | Token counts | Model-visible history | | --------------------------------- | :---------------: | :----------: | :-------------------: | | `anyswe_agent` | ❌ | ❌ | ❌ | | `anyterminal_agent` | ✅ | ✅ | ✅ | | `aviary_agent` | ✅ | ✅ | ✅ | | `browsecomp_agent` | ✅ | ✅ | ✅ | | `claude_code_agent` | ✅ | ✅ | ✅ | | `cline_agent` | ✅ | ✅ | ✅ | | `codex_agent` | ✅ | ✅ | ✅ | | `critpt_agent` | ✅ | ✅ | ✅ | | `cvdp_agent` | 🟡 | 🟡 | 🟡 | | `finance_agent` | ❌ | 🟡 | ❌ | | `gymnasium_agent` | ✅ | ✅ | ✅ | | `harbor_agent` | ❌ | ❌ | ❌ | | `hermes_agent` | ✅ | ✅ | ✅ | | `kilocode_agent` | ✅ | ✅ | ✅ | | `labbench2_vlm_agent` | ✅ | ✅ | 🟡 | | `langgraph_agent` | ❌ | ❌ | ❌ | | `mini_swe_agent` | ❌ | ❌ | ❌ | | `mini_swe_agent_2` | ✅ | ✅ | ✅ | | `non_executing_simple_agent` | ✅ | ✅ | ✅ | | `openclaw_agent` | ✅ | ✅ | 🟡 | | `opencode_agent` | ✅ | ✅ | ✅ | | `opencode_sandboxed_agent` | ✅ | ✅ | ✅ | | `osworld_agent` | 🟡 | 🟡 | 🟡 | | `pi_agent` | ✅ | ✅ | ✅ | | `proof_refinement_agent` | ✅ | ✅ | ✅ | | `remote_agent` | ❌ | 🟡 | ❌ | | `scicode_agent` | ❌ | ❌ | ❌ | | `simple_agent` | ✅ | ✅ | ✅ | | `speed_bench_agent` | ✅ | ✅ | ✅ | | `stirrup_agent` | 🟡 | 🟡 | 🟡 | | `swe_agents` / OpenCode (legacy) | ✅ | ✅ | ✅ | | `swe_agents` / OpenHands (legacy) | ❌ | ❌ | 🟡 | | `tau2` | ✅ | ✅ | ✅ | | `tool_simulation_agent` | ✅ | ✅ | ✅ | | `toolsandbox_agent` | ✅ | ✅ | ✅ | | `verifiers_agent` | ✅ | ✅ | ✅ | ## Agent Activity Turns require explicit semantic-turn records. Tool execution covers output, status, and timestamps; parallel timing requires independent tool intervals. Resource/sandbox coverage means that the benchmark path emits standardized agent evidence, not that every resource metric is available. | Agent | Semantic turns | Tool execution | Parallel timing | Resource/sandbox path | | --------------------------------- | :------------: | :------------: | :-------------: | :-------------------: | | `claude_code_agent` | ❌ | ✅ | ✅ | ✅ | | `hermes_agent` | ❌ | ✅ | ✅ | ✅ | | `labbench2_vlm_agent` | 🟡 | 🟡 | ❌ | ✅ | | `openclaw_agent` | ❌ | ✅ | ✅ | ✅ | | `opencode_agent` | ❌ | ✅ | ✅ | ✅ | | `opencode_sandboxed_agent` | ❌ | ✅ | ✅ | ✅ | | `pi_agent` | ❌ | ✅ | ✅ | ✅ | | `simple_agent` | 🟡 | 🟡 | ❌ | ✅ | | `swe_agents` / OpenCode (legacy) | ❌ | ❌ | ❌ | ✅ | | `swe_agents` / OpenHands (legacy) | ❌ | ❌ | ❌ | ✅ | The other agents listed above report these four dimensions as unavailable. Model HTTP capture alone does not establish tool execution or sandbox activity. ## Path Qualifications | Path | Qualification | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | CVDP | Partial capture on the Simple path | | Finance and Remote | Aggregate usage only; no standardized per-call metadata | | Simple | Resolution can be missing; raised tool failures can be omitted | | LabBench | Resolution and tool evidence can be partial; `multimodal_history_redacted` omits trajectory payload copies | | OSWorld | M3 and Pointer routes call providers directly | | Stirrup | Calls outside the policy path are not fully captured | | OpenClaw | Includes standalone resources and PinchBench sandbox paths; history is partial | | OpenCode | Includes standalone and sandboxed agents; the sandboxed path combines harness and verifier-sandbox evidence | | Legacy OpenHands | One cumulative root conversation; pinned fork bypasses rollout-prefixed capture | | Verifiers | Gym Model Server path captures environment-initiated calls; no agent observations or tool status/timing; agent-turn and bound-call health checks remain unobserved | Inspect `ng_trajectory.gaps` before relying on partial coverage. Aggregate-metrics requests omit the trajectory and source attachments. ## ATIF export `gym eval export --format atif` converts the supported `ng_trajectory` version `1.0` subset into ATIF v1.7 without loading NeMo Relay. The initial converter accepts a completed root invocation with one initial system/user input phase, exactly one captured model call per turn, text-only output (including ordered multipart tool results), and unambiguous function-call and function-result relationships. Model calls may be joined by `model_call_id` or by a unique `(model_ref, response_id)` pair. It rejects later user/system turns and source records when their status, evidence gaps, content type, invocation structure, tool correlation, or provider completion evidence would require dropping or guessing information. This strict boundary means an accepted conversion is complete for the supported source profile; it does not claim that the original agent harness observed every action outside Gym. Responses message `phase` values such as `commentary` and `final_answer` are rejected because ATIF v1.7 has no equivalent field. Namespaced Responses function calls are also rejected because the standard ATIF tool-call fields cannot retain the separate namespace identity. A missing or null phase or namespace is accepted because it carries no additional source meaning. Provider-native hosted tools, shell and patch calls, tool search, compaction records, approvals, and additional-tool records remain outside the initial generic function-call profile and are rejected rather than flattened. The semantic profile maps ordered messages, reasoning, function calls and results, turn identity and resolution, tool timing, normalized model identity, and normalized token counts. Training token IDs, log probabilities, and routed-expert metadata are outside the initial profile and fail conversion rather than being copied into an extension. Captured provider request and response values are retained opaquely under step-level `extra.nemo_gym.model_call` metadata because ATIF's standard step fields do not carry those provider-specific payloads. The converter does not parse those raw payloads or use them to reconstruct the canonical trajectory; the producer-owned `response_metadata` and `token_stats` fields supply completion, model, and usage evidence. Per-step token fields are preserved when available. Final prompt, completion, and cached-token totals are emitted only when every model call provides the corresponding field, so unavailable values are never counted as zero. Each agent step retains the normalized model identity. The root agent model is populated only when every model-backed step uses the same known model; routed or fallback trajectories keep their model identities on the individual steps instead. The strict ATIF reverify path additionally requires complete prompt and completion counts when any supported usage is present because one Gym response cannot represent partial standard usage. A partial-metric export remains valid ATIF for other consumers, but Gym reverify rejects it instead of dropping the known counts or reporting a false complete aggregate. The ATIF `agent.name` comes from each rollout row's `agent_ref.name`. The initial batch exporter accepts one agent identity per input file because the caller supplies one corresponding agent implementation version. Conversion provenance is kept under the ATIF root's `extra.nemo_gym` namespace rather than overloading the agent identity fields: | Metadata | Value | Purpose | | ------------------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | Exporter | NeMo Gym and its package version | Identifies the implementation that created the ATIF file. | | Source | Gym rollout and `ng_trajectory` schema version | Records the format that was converted. | | Conversion status | `complete` | Confirms that the supported normalized `ng_trajectory` profile was mapped without omission; it does not attest to the contents of opaque provider payloads. | | Conversion warnings | Not emitted by strict v1 | Reserved for a future best-effort mode that can describe information it could not represent. | The initial strict converter does not emit partial files or conversion warnings. Unsupported or lossy inputs fail validation. A future best-effort mode would need an explicit `partial` status and structured warnings rather than reusing `complete`. ### Terminus2 decisions and invocation ownership With model-call capture enabled as described above, `terminus_2_sandboxed_agent` emits an `AgentInvocation` for each identified rollout execution. Its execution-specific `x-session-id` header lets the collector link captured model calls to that invocation, including retries without a response ID and summarization calls. The adapter records `TrajectoryTurn` entries at Harbor's main-agent decision boundary, before terminal commands execute. Each observed decision retains the exact input sent for its selected response, the answer and reasoning, and an exact model-server/response-ID reference. Its timestamp is the observed response time. `turn_no` counts main decision attempts; `step_count` counts completed nonempty terminal-command batch attempts before that decision. A returned batch timeout counts as an attempt, not proof that every command ran. One decision can request several commands. `resolved` remains unset because a model's completion claim does not establish task success. Decisions remain available after parser errors, tool failures, or cancellation. Responses rejected by the adapter and synthetic Harbor fallback messages are not presented as accepted main-agent decisions. Their available model-call evidence remains invocation-owned, with diagnostic gaps. Known summarization boundaries also emit `ContextCompactionObservation` records with exact response references; auxiliary fallback calls are not mislabeled as main decisions. Missing or reused response IDs produce gaps instead of guessed associations. `turns_unavailable` is reported only when no actual decision response could be represented. Observed turns enable `rollout_missing_agent_turns` and `agent_turn_hollow`. Invocation ownership enables `model_call_failed`, `model_call_missing_token_counts`, `model_call_zero_completion_tokens`, `model_call_runaway_generation`, and `trajectory_capture_mismatch` under their documented capture and payload requirements. See [Rollout health checks](/evaluation/rollout-health). The underlying HTTP transport can retry after server admission without notifying the adapter. Complete turn-call accounting therefore remains unavailable: `transport_retry_visibility_unavailable` and `turn_model_call_scope_incomplete` keep rollout token reconciliation and full policy-call coverage unobserved while preserving those seven checks. This producer does not establish an overall healthy rollout or task success from calls.