Trajectory Capability Matrix

Use this reference to check which evidence an agent path provides. Start with Rollout Evidence for collection and artifact relationships.

✅ Full means complete for the documented path; 🟡 Partial means some fields or paths are missing; ❌ Unavailable means the path does not supply that evidence. These are manually maintained coverage claims, not guarantees for every configuration.

The statuses below retain the coverage reported in main at 88d9d453. That source does not record a full runtime-validation date or per-row test configuration. Contributors changing an agent should update its row with the config, execution path, and validation evidence in the same pull request.

Model Calls and History

Capture requires observability_enabled, a shared model_call_capture_dir, and a rollout-prefixed Gym Model Server URL. Direct-provider routes bypass capture. Token counts depend on provider usage fields. History may come from captured payloads or harness conversations.

Agent	Per-call metadata	Token counts	Model-visible history
`anyswe_agent`	❌	❌	❌
`anyterminal_agent`	✅	✅	✅
`aviary_agent`	✅	✅	✅
`browsecomp_agent`	✅	✅	✅
`claude_code_agent`	✅	✅	✅
`cline_agent`	✅	✅	✅
`codex_agent`	✅	✅	✅
`critpt_agent`	✅	✅	✅
`cvdp_agent`	🟡	🟡	🟡
`finance_agent`	❌	🟡	❌
`gymnasium_agent`	✅	✅	✅
`harbor_agent`	❌	❌	❌
`hermes_agent`	✅	✅	✅
`kilocode_agent`	✅	✅	✅
`labbench2_vlm_agent`	✅	✅	🟡
`langgraph_agent`	❌	❌	❌
`mini_swe_agent`	❌	❌	❌
`mini_swe_agent_2`	✅	✅	✅
`non_executing_simple_agent`	✅	✅	✅
`openclaw_agent`	✅	✅	🟡
`opencode_agent`	✅	✅	✅
`opencode_sandboxed_agent`	✅	✅	✅
`osworld_agent`	🟡	🟡	🟡
`pi_agent`	✅	✅	✅
`proof_refinement_agent`	✅	✅	✅
`remote_agent`	❌	🟡	❌
`scicode_agent`	❌	❌	❌
`simple_agent`	✅	✅	✅
`speed_bench_agent`	✅	✅	✅
`stirrup_agent`	🟡	🟡	🟡
`swe_agents` / OpenCode (legacy)	✅	✅	✅
`swe_agents` / OpenHands (legacy)	❌	❌	🟡
`tau2`	✅	✅	✅
`tool_simulation_agent`	✅	✅	✅
`toolsandbox_agent`	✅	✅	✅
`verifiers_agent`	✅	✅	✅

Agent Activity

Turns require explicit semantic-turn records. Tool execution covers output, status, and timestamps; parallel timing requires independent tool intervals. Resource/sandbox coverage means that the benchmark path emits standardized agent evidence, not that every resource metric is available.

Agent	Semantic turns	Tool execution	Parallel timing	Resource/sandbox path
`claude_code_agent`	❌	✅	✅	✅
`hermes_agent`	❌	✅	✅	✅
`labbench2_vlm_agent`	🟡	🟡	❌	✅
`openclaw_agent`	❌	✅	✅	✅
`opencode_agent`	❌	✅	✅	✅
`opencode_sandboxed_agent`	❌	✅	✅	✅
`pi_agent`	❌	✅	✅	✅
`simple_agent`	🟡	🟡	❌	✅
`swe_agents` / OpenCode (legacy)	❌	❌	❌	✅
`swe_agents` / OpenHands (legacy)	❌	❌	❌	✅

The other agents listed above report these four dimensions as unavailable. Model HTTP capture alone does not establish tool execution or sandbox activity.

Path Qualifications

Path	Qualification
CVDP	Partial capture on the Simple path
Finance and Remote	Aggregate usage only; no standardized per-call metadata
Simple	Resolution can be missing; raised tool failures can be omitted
LabBench	Resolution and tool evidence can be partial; `multimodal_history_redacted` omits trajectory payload copies
OSWorld	M3 and Pointer routes call providers directly
Stirrup	Calls outside the policy path are not fully captured
OpenClaw	Includes standalone resources and PinchBench sandbox paths; history is partial
OpenCode	Includes standalone and sandboxed agents; the sandboxed path combines harness and verifier-sandbox evidence
Legacy OpenHands	One cumulative root conversation; pinned fork bypasses rollout-prefixed capture
Verifiers	Gym Model Server path captures environment-initiated calls; no agent observations or tool status/timing; agent-turn and bound-call health checks remain unobserved

Inspect ng_trajectory.gaps before relying on partial coverage. Aggregate-metrics requests omit the trajectory and source attachments.

Terminus2 decisions and invocation ownership

With model-call capture enabled as described above, terminus_2_sandboxed_agent emits an AgentInvocation for each identified rollout execution. Its execution-specific x-session-id header lets the collector link captured model calls to that invocation, including retries without a response ID and summarization calls.

The adapter records TrajectoryTurn entries at Harbor’s main-agent decision boundary, before terminal commands execute. Each observed decision retains the exact input sent for its selected response, the answer and reasoning, and an exact model-server/response-ID reference. Its timestamp is the observed response time. turn_no counts main decision attempts; step_count counts completed nonempty terminal-command batch attempts before that decision. A returned batch timeout counts as an attempt, not proof that every command ran. One decision can request several commands. resolved remains unset because a model’s completion claim does not establish task success.

Decisions remain available after parser errors, tool failures, or cancellation. Responses rejected by the adapter and synthetic Harbor fallback messages are not presented as accepted main-agent decisions. Their available model-call evidence remains invocation-owned, with diagnostic gaps. Known summarization boundaries also emit ContextCompactionObservation records with exact response references; auxiliary fallback calls are not mislabeled as main decisions. Missing or reused response IDs produce gaps instead of guessed associations. turns_unavailable is reported only when no actual decision response could be represented.

Observed turns enable rollout_missing_agent_turns and agent_turn_hollow. Invocation ownership enables model_call_failed, model_call_missing_token_counts, model_call_zero_completion_tokens, model_call_runaway_generation, and trajectory_capture_mismatch under their documented capture and payload requirements. See Rollout health checks.

The underlying HTTP transport can retry after server admission without notifying the adapter. Complete turn-call accounting therefore remains unavailable: transport_retry_visibility_unavailable and turn_model_call_scope_incomplete keep rollout token reconciliation and full policy-call coverage unobserved while preserving those seven checks. This producer does not establish an overall healthy rollout or task success from calls.