> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo/gym/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo/gym/_mcp/server. # Trajectory capability matrix > Observed trajectory coverage by criterion and producer # Trajectory capability matrix This matrix reports the trajectory evidence that current producers emit for the evaluated agent paths. It describes observed producer coverage, not everything that the schema can represent or an unconditional guarantee for every way of running an agent. `ng_trajectory` schema version `1.0` combines available model-call capture and agent observations in a rollout record. Model-call capture records qualifying calls routed through the Gym Model Server, while agent observations provide harness-side turns and tool activity. The trajectory joins the available evidence from both sources. Direct-provider calls bypass Gym Model Server capture, so the trajectory cannot obtain Gym-captured model-call evidence for those calls even when agent observations are available. The trajectory can contain task and rollout identity, model calls and token usage, semantic turns, invocation-scoped model-visible histories, tool attempts, resolution, step count, and explicit evidence gaps. Source attachments remain under `ng_model_call_capture` and `ng_agent_observations`. ## Key caveats * Coverage is path-dependent. A producer may use both captured Gym Model Server calls and uncaptured direct-provider calls. * Model-call evidence requires `observability_enabled: true`, `model_call_capture_dir: /data/model-calls`, and calls through a `/ng-rollout/` Gym Model Server prefix. See [Model-call capture](/model-server/model-call-capture). * `O` and `X` describe current producer output, not fields that the `ng_trajectory` schema is unable to represent. * Token fields are nullable, and provider-reported details are preserved only when available. LabBench rows with the `multimodal_history_redacted` gap omit captured request and response payload copies from `ng_trajectory`. * Aggregate-metrics requests omit `ng_trajectory`, `ng_model_call_capture`, and `ng_agent_observations` attachments. See [Model-call capture](/model-server/model-call-capture) for payload retention and access-control requirements. Token fields are `prompt_tokens`, `completion_tokens`, `reasoning_tokens`, `total_tokens`, and `cached_tokens`. ## Acceptance criteria | ID | Criterion | | -- | ------------------------------------------------------------------------------------------------------------------- | | C1 | Standard schema for per-model-call token statistics and response metadata | | C2 | Prompt, completion, reasoning, total, and cached token counts when available | | C3 | Task and rollout identity, turn number, timestamp, question, answer, reasoning, resolution, and step count per turn | | C4 | Model-visible input and output history can be reconstructed from persisted rollout JSONL | | C5 | Tool output, status, start and completion timestamps, and duration | | C6 | Independent timing for parallel tool calls | | C7 | Resource-server-backed and custom or sandbox-backed benchmarks | ## Agent coverage * `V`: complete for the evaluated path under the documented observability configuration. * `O`: currently partial or path-dependent. * `X`: currently unavailable. In practice, rely on `V` for the evaluated path when observability is configured as documented, inspect configuration and trajectory gaps before relying on `O`, and do not expect the producer to supply `X` on the evaluated path. These symbols are not absolute agent capabilities: alternate execution paths can produce different evidence. For C1, C2, and C4, `V` evaluates the correlated Gym Model Server path; direct-provider alternatives are outside capture. | Agent | C1 | C2 | C3 | C4 | C5 | C6 | C7 | | --------------------------------- | :-: | :-: | :-: | :-: | :-: | :-: | :-: | | `anyswe_agent` | X | X | X | X | X | X | X | | `anyterminal_agent` | V | V | X | V | X | X | X | | `aviary_agent` | V | V | X | V | X | X | X | | `browsecomp_agent` | V | V | X | V | X | X | X | | `claude_code_agent` | V | V | X | V | V | V | V | | `cline_agent` | V | V | X | V | X | X | X | | `codex_agent` | V | V | X | V | X | X | X | | `critpt_agent` | V | V | X | V | X | X | X | | `cvdp_agent` | O | O | X | O | X | X | X | | `finance_agent` | X | O | X | X | X | X | X | | `gymnasium_agent` | V | V | X | V | X | X | X | | `harbor_agent` | X | X | X | X | X | X | X | | `hermes_agent` | V | V | X | V | V | V | V | | `kilocode_agent` | V | V | X | V | X | X | X | | `labbench2_vlm_agent` | V | V | O | O | O | X | V | | `langgraph_agent` | X | X | X | X | X | X | X | | `mini_swe_agent` | X | X | X | X | X | X | X | | `mini_swe_agent_2` | V | V | X | V | X | X | X | | `non_executing_simple_agent` | V | V | X | V | X | X | X | | `openclaw_agent` | V | V | X | O | V | V | V | | `opencode_agent` | V | V | X | V | V | V | V | | `opencode_sandboxed_agent` | V | V | X | V | V | V | V | | `osworld_agent` | O | O | X | O | X | X | X | | `pi_agent` | V | V | X | V | V | V | V | | `proof_refinement_agent` | V | V | X | V | X | X | X | | `remote_agent` | X | O | X | X | X | X | X | | `scicode_agent` | X | X | X | X | X | X | X | | `simple_agent` | V | V | O | V | O | X | V | | `speed_bench_agent` | V | V | X | V | X | X | X | | `stirrup_agent` | O | O | X | O | X | X | X | | `swe_agents` / OpenCode (legacy) | V | V | X | V | X | X | V | | `swe_agents` / OpenHands (legacy) | X | X | X | O | X | X | V | | `tau2` | V | V | X | V | X | X | X | | `tool_simulation_agent` | V | V | X | V | X | X | X | | `toolsandbox_agent` | V | V | X | V | X | X | X | | `verifiers_agent` | X | X | X | X | X | X | X | ### Coverage notes * Simple and LabBench C3 support is partial because verifier responses may omit resolution status. * Other `O` values reflect the following paths or evidence gaps: * The CVDP Simple path. * Finance and Remote aggregate usage. * LabBench image redaction. * OSWorld M3 and Pointer direct-provider routes. * Raised failures omitted by Simple-derived agents. * Stirrup calls outside its policy path. * OpenClaw coverage includes its standalone resource-server path and the PinchBench sandbox benchmark path. * OpenCode coverage includes both the standalone producer and the decoupled `opencode_sandboxed_agent` path. The decoupled path composes agent observations with verifier-sandbox evidence returned by its resources server. * Legacy OpenHands retains one cumulative root conversation, so its model-visible history remains partial. Its pinned fork does not route model calls through a rollout-prefixed Gym Model Server endpoint. * C7 requires standardized agent-side trajectory evidence; model HTTP capture alone does not satisfy it.