Trajectory capability matrix

This matrix reports the trajectory evidence that current producers emit for the evaluated agent paths. It describes observed producer coverage, not everything that the schema can represent or an unconditional guarantee for every way of running an agent.

ng_trajectory schema version 1.0 combines available model-call capture and agent observations in a rollout record. Model-call capture records qualifying calls routed through the Gym Model Server, while agent observations provide harness-side turns and tool activity. The trajectory joins the available evidence from both sources. Direct-provider calls bypass Gym Model Server capture, so the trajectory cannot obtain Gym-captured model-call evidence for those calls even when agent observations are available.

The trajectory can contain task and rollout identity, model calls and token usage, semantic turns, invocation-scoped model-visible histories, tool attempts, resolution, step count, and explicit evidence gaps. Source attachments remain under ng_model_call_capture and ng_agent_observations.

Key caveats

• Coverage is path-dependent. A producer may use both captured Gym Model Server calls and uncaptured direct-provider calls.
• Model-call evidence requires observability_enabled: true, model_call_capture_dir: /data/model-calls, and calls through a /ng-rollout/<rollout_id> Gym Model Server prefix. See Model-call capture.
• O and X describe current producer output, not fields that the ng_trajectory schema is unable to represent.
• Token fields are nullable, and provider-reported details are preserved only when available. LabBench rows with the multimodal_history_redacted gap omit captured request and response payload copies from ng_trajectory.
• Aggregate-metrics requests omit ng_trajectory, ng_model_call_capture, and ng_agent_observations attachments.

See Model-call capture for payload retention and access-control requirements. Token fields are prompt_tokens, completion_tokens, reasoning_tokens, total_tokens, and cached_tokens.

Acceptance criteria

Agent coverage

• V: complete for the evaluated path under the documented observability configuration.
• O: currently partial or path-dependent.
• X: currently unavailable.

In practice, rely on V for the evaluated path when observability is configured as documented, inspect configuration and trajectory gaps before relying on O, and do not expect the producer to supply X on the evaluated path. These symbols are not absolute agent capabilities: alternate execution paths can produce different evidence. For C1, C2, and C4, V evaluates the correlated Gym Model Server path; direct-provider alternatives are outside capture.

Coverage notes

• Simple and LabBench C3 support is partial because verifier responses may omit resolution status.
• Other O values reflect the following paths or evidence gaps:
  • The CVDP Simple path.
  • Finance and Remote aggregate usage.
  • LabBench image redaction.
  • OSWorld M3 and Pointer direct-provider routes.
  • Raised failures omitted by Simple-derived agents.
  • Stirrup calls outside its policy path.
• OpenClaw coverage includes its standalone resource-server path and the PinchBench sandbox benchmark path.
• Verifiers C1, C2, and C4 cover the Gym Model Server path: the agent routes the client it hands to verifiers through the rollout capture prefix, so calls the environment makes on its own are captured and correlated. C5 and C6 are unavailable because the agent emits no ng_agent_observations, so tool attempts carry no status or timing, and rollout-health checks that read agent turns or bound calls stay unobserved.
• OpenCode coverage includes both the standalone producer and the decoupled opencode_sandboxed_agent path. The decoupled path composes agent observations with verifier-sandbox evidence returned by its resources server.
• Legacy OpenHands retains one cumulative root conversation, so its model-visible history remains partial. Its pinned fork does not route model calls through a rollout-prefixed Gym Model Server endpoint.
• C7 requires standardized agent-side trajectory evidence; model HTTP capture alone does not satisfy it.

Terminus2 decisions and invocation ownership

With model-call capture enabled as described above, terminus_2_sandboxed_agent emits an AgentInvocation for each identified rollout execution. Its execution-specific x-session-id header lets the collector link captured model calls to that invocation, including retries without a response ID and summarization calls.

The adapter records TrajectoryTurn entries at Harbor’s main-agent decision boundary, before terminal commands execute. Each observed decision retains the exact input sent for its selected response, the answer and reasoning, and an exact model-server/response-ID reference. Its timestamp is the observed response time. turn_no counts main decision attempts; step_count counts completed nonempty terminal-command batch attempts before that decision. A returned batch timeout counts as an attempt, not proof that every command ran. One decision can request several commands. resolved remains unset because a model’s completion claim does not establish task success.

Decisions remain available after parser errors, tool failures, or cancellation. Responses rejected by the adapter and synthetic Harbor fallback messages are not presented as accepted main-agent decisions. Their available model-call evidence remains invocation-owned, with diagnostic gaps. Known summarization boundaries also emit ContextCompactionObservation records with exact response references; auxiliary fallback calls are not mislabeled as main decisions. Missing or reused response IDs produce gaps instead of guessed associations. turns_unavailable is reported only when no actual decision response could be represented.

Observed turns enable rollout_missing_agent_turns and agent_turn_hollow. Invocation ownership enables model_call_failed, model_call_missing_token_counts, model_call_zero_completion_tokens, model_call_runaway_generation, and trajectory_capture_mismatch under their documented capture and payload requirements. See Rollout health checks.

The underlying HTTP transport can retry after server admission without notifying the adapter. Complete turn-call accounting therefore remains unavailable: transport_retry_visibility_unavailable and turn_model_call_scope_incomplete keep rollout token reconciliation and full policy-call coverage unobserved while preserving those seven checks. This producer does not establish an overall healthy rollout or task success from calls.