Skip to navigation

Results and Errors

View as MarkdownOpen in Claude

For examples that reuse config, start with Start With One Run.

Unified Run Results

Adapters report completed invocations through a normalized RunResult, including harness failures they can normalize. If the adapter host crashes or the protocol fails before a normalized result is available, the SDK can raise FabricRuntimeError instead. Inspect status, error, events, and artifacts first, then process output when the status is successful.

Important fields:

FieldMeaning
statusTerminal invocation status such as success, failure, or cancellation.
outputHarness output normalized to the configured output schema.
errorStructured failure metadata when available.
usageOptional RunUsage with invocation-local input, cached-input, output, and total token counts, reported cost in USD, and provider metadata.
system_statsOptional InvocationSystemStats: monotonic adapter exchange duration, before/after observations with availability, and an optional CPU counter difference estimate.
resource_snapshotOptional RuntimeResourceSnapshot of the runtime, read once after the adapter returned the result.
artifactsOutput files, logs, patches, native artifacts, and other materialized references.
telemetryReferences to NeMo Relay or other telemetry streams produced by the run.
eventsOrdered normalized lifecycle and invocation events.
metadataResult-specific structured metadata.
runtime_id, invocation_id, request_idIDs for correlation across runtimes, logs, telemetry, and artifacts.

These are structured correlation fields, not interchangeable metadata: runtime_id identifies the runtime lifecycle, invocation_id identifies one invocation within that runtime, and request_id correlates the caller’s request. NeMo Fabric-generated values use type-specific prefixes such as runtime-, invocation-, and request-; callers may provide their own request_id. Consumers should store and log each field separately and otherwise treat its value as opaque rather than parsing the identifier encoding.

Usage is absent when the target does not report it. Read the typed values only when usage is available:

if result.usage is not None:
print(result.usage.get("input_tokens"))
print(result.usage.get("cached_input_tokens"))
print(result.usage.get("input_tokens_include_cache"))
print(result.usage.get("output_tokens"))
print(result.usage.get("total_tokens"))
print(result.usage.get("cost_usd"))

Each counter is optional: missing values are unknown, not zero. cached_input_tokens reports cached prompt tokens. input_tokens_include_cache is True when those tokens are already included in input_tokens, False when they are excluded, and absent when the adapter cannot establish the semantics. Consumers must not add cached tokens to an inclusive input count. Adapters can report cache-write, reasoning, peak request, and per-model detail through the descriptor-declared usage.extensions["nemo_fabric.detailed_token_usage.v1"] object. NeMo Fabric preserves that object at RunUsage.metadata["nemo_fabric.detailed_token_usage.v1"]. Consumers treat detailed input and output independently: any detailed accounting field for a dimension makes that dimension authoritative, with no fallback to its top-level counters. The views can diverge, and an inclusive total remains unknown when the selected detailed counters do not provide enough information to calculate it. cost_usd contains reported cost only; keep estimates separately in provider metadata rather than presenting them as measured charges.

If NeMo Fabric cannot resolve config, start a runtime, or obtain a normalized result, the SDK raises a FabricError subclass instead of returning a partial RunResult.

Events

Each RunResult includes the normalized events collected for that invocation. Event kinds and detail may vary by adapter, but their lifecycle and correlation fields use the common NeMo Fabric contract.

Events are useful for:

  • rendering invocation history in application or service UIs;
  • forwarding logs and status to evaluation harnesses;
  • correlating runtime, invocation, adapter, and telemetry IDs;
  • reporting structured failures alongside the terminal result.

Response and Artifact Limits

runtime.response_limits applies to buffered invocation results on the host and inside OpenShell. Local enforcement requires an explicit response_limits value; OpenShell applies default transfer budgets when it is omitted. The SDK enforces configured budgets independently of the harness; adapters do not need a capability declaration for them.

from nemo_fabric import ResponseLimits
config.runtime.response_limits = ResponseLimits(
max_response_bytes=8 * 1024 * 1024,
max_artifact_bytes=16 * 1024 * 1024,
max_artifact_files=32,
max_artifact_file_bytes=8 * 1024 * 1024,
)

The equivalent YAML belongs in the normalized runtime section:

runtime:
response_limits:
max_response_bytes: 8388608
max_artifact_bytes: 16777216
max_artifact_files: 32
max_artifact_file_bytes: 8388608
FieldDefaultWhat is counted
max_response_bytes4 MiBCompact UTF-8 JSON serialization of the normalized invocation RunResult, excluding transport envelopes and the trailing newline. Convenience runs recheck after stop events or stop errors are added.
max_artifact_bytes2 MiBSum of the uncompressed sizes of unique declared artifact files.
max_artifact_files16Unique declared artifact paths; repeated references count once.
max_artifact_file_bytes1 MiBUncompressed size of each declared artifact file.

Limits are inclusive: a value exactly at the budget passes. Each supplied field must be an integer from 1 through 4294967295. Omit individual fields to keep their defaults. Omit response_limits or use None to preserve existing local-host behavior without SDK response or artifact budget enforcement. OpenShell retains the default transfer budgets when limits are omitted. Set response_limits={} to opt into all default budgets on the local host. Unknown limit names are rejected. File and aggregate byte budgets are independent, so raising one does not raise the other. SDK diagnostics under .fabric/, workspace patches, and workspace status files do not consume the declared-agent artifact budgets. On the host, missing files and non-file references contribute no artifact bytes; their declared paths still count toward the path budget. OpenShell collection requires readable regular files. Provider transport allowances scale with these budgets; compression does not add a separate screenshot size limit. The 32-bit schema ceiling is not a guaranteed transfer capacity: OpenShell buffers compressed artifacts as JSON byte arrays, which increase wire size and memory use, and collection retains a separate 30-second deadline. Raising byte budgets does not increase sandbox memory or that deadline. Large batches can still fail with gateway_timeout or artifact_read_failed; split them into smaller batches or return references to externally stored data when they exceed the sandbox’s transfer capacity.

A breach returns status == "failed" with error.code == "response_limit_exceeded". The error names the limit, configured value, observed value and offending asset in both its message and metadata. The rejected output and artifact manifest are cleared, along with adapter metadata and events. Evidence about completed work remains available: usage, execution, telemetry, system_stats, and resource_snapshot are preserved. The result remains failed; payloads are not silently truncated. Only the SDK’s canonical limit diagnostic and retained execution evidence are exempt from the response budget, so even a one-byte budget reports why it failed. Additional payloads or shutdown metadata are checked normally. A limit failure does not undo agent side effects and is not automatically retried.

For a large screenshot, keep the image in a declared artifact instead of embedding base64 in the response. Raise both max_artifact_file_bytes and max_artifact_bytes enough to hold the complete file and the other artifacts, then invoke again. On success, read its returned ArtifactRef.path; OpenShell copies the complete bytes into the host runtime.artifacts destination. Local execution keeps the original file path. The result must still fit its response budget after artifact paths and execution metadata are added on the host. These are returned-result budgets, not limits on intermediate model messages or streaming chunks already delivered to a consumer.

Errors

All public SDK errors inherit from FabricError. One-shot lifecycle failures can retain obtained runtime and request identities, cleanup events, and already-selected software evidence in error.details["runtime_evidence"]. These observations do not establish a terminal invocation result; unknown usage remains unknown. The original exception and its cause remain authoritative.

ErrorMeaning
FabricConfigErrorInvalid config, request, or override.
FabricCapabilityErrorSelected adapter does not support the requested operation.
FabricRuntimeErrorRuntime startup, invocation, or shutdown failed before a normalized result could be returned.
FabricStateErrorInvalid runtime state transition, such as invoking after stop or starting overlapping invocations.
FabricNativeUnavailableErrorNative extension is not installed or importable.
ResourceObservationUnavailableErrorRuntime.resource_snapshot() cannot read the runtime because it stopped, is stopping, its adapter host exited, a live process in its tree or the process table cannot be read, or the process tree kept changing during the read. Subclass of FabricRuntimeError.
ResourceObservationUnsupportedErrorThe runtime’s adapter kind, environment provider, or operating system has no resource reader. Subclass of FabricRuntimeError.

Consumers own job-level retries and rollout-level failure policy. runtime.timeout_seconds sets a positive, finite total invocation limit. Omit it or set it to None to disable NeMo Fabric’s total invocation deadline. The same configuration applies whether the agent executes on the host or inside a sandbox such as OpenShell. runtime.max_turns independently limits harness turns where supported. Startup, health checks, and cleanup retain separate deadlines; harness-native limits still apply.

Core host-operation deadlines (host_timeout) become Python TimeoutError, then FabricRuntimeError with code == "timeout" and the failed lifecycle stage. If invocation succeeds in Fabric.run() but stop reaches this deadline, it returns a failed RunResult with error.stage == "stop" and error.code == "timeout". Check the returned result as well as raised errors. Adapter lifecycle failures with canonical code == "timeout" also preserve deadline identity. Raised FabricRuntimeError.details retains structured source metadata: runtime host deadlines report owner: "fabric_runtime", timeout_seconds, and cause_code: "host_timeout"; adapter deadlines retain their adapter-owned limit and cause. Native diagnostic messages remain available through exception chaining.

OpenShell preserves this deadline identity across the guest SDK and provider boundary. Only bounded owner, timeout_seconds, deadline_millis, cause_code, and native_code details are forwarded; arbitrary guest metadata and native error text are excluded.

Adapters can report execution deadlines in normalized results with result.error.code == "timeout". Other adapter-specific timeout codes are not automatically reclassified. Diagnostic text is not a timeout classifier. Other adapter lifecycle failures raise FabricRuntimeError with the adapter’s code, such as openclaw_gateway_start_timeout. Single-invocation runs attempt to stop the runtime before returning. A Runtime used with async with also attempts cleanup after an invocation error; if cleanup fails, that failure is attached to the original exception rather than replacing it. NeMo Fabric records structured error metadata when possible and returns enough detail for the consumer to decide what to do next.

Recover a buffered invocation result

NeMo Fabric saves each completed RunResult as <artifact root>/.fabric/<runtime_id>/<invocation_id>/run-result.json before returning the result. On Unix, result and request-record files are readable only by their owner. The artifact root comes from runtime.artifacts or the environment; without either setting, NeMo Fabric uses its default artifact directory. Files remain until their storage is removed; no automatic retention or eviction is added. The owning Fabric client retains the artifact location and environment identity in its bounded stopped-runtime records, so that client can look up results after Runtime.stop() without repeating the artifact path. After record eviction or from a new client, provide the original explicit artifact root unless the default root applies.

On Unix, default storage uses ~/.cache/nemo-fabric-<effective user ID> when the user-owned home directory and resolved cache path pass the ownership and ancestry checks. Local host directories use the same protected root, including when an artifact root is configured explicitly. If the home or cache path fails validation, NeMo Fabric resolves temporary-directory symlinks, validates the resolved ancestry, and uses <temp_dir>/nemo-fabric-<effective user ID>-private with mode 0700. An existing fallback directory must belong to that user, have private permissions, and not be a symlink. Native writes and SDK lookup select the same root. Local one-shot results remain unavailable until the shutdown outcome is saved. A failed final save or interruption keeps the request reserved and prevents lookup from returning an earlier success. Rust RunRequest::default() generates a request ID before dispatch; explicitly supplied blank IDs are rejected.

Use a trusted artifact directory. On Unix, result storage validates ownership, permissions, and symlinks within the private .fabric subtree, and requires private record-file permissions. Explicit caller-configured artifact roots can have shared or group-writable ancestors. Stored results have a default size limit of 4 MiB. An explicitly configured larger runtime.response_limits.max_response_bytes raises that storage limit for the invocation; lookup uses the limit retained in its private request record. Request identifiers must not be blank or contain NUL. Buffered retries can return the saved result; a request ID cannot switch between buffered and streaming invocations. A completed streaming request returns stream_replay_unsupported on replay; retrieve its terminal result with invocation_result() instead.

The SDK keeps the runtime active after request_conflict, invocation_pending, and stream_replay_unsupported. These request rejections do not dispatch a new turn.

Choose a stable RunRequest.request_id before submitting work. After a command timeout, invocation_timeout reports unknown effects and returns the runtime and request identifiers. The invocation can continue. Retrieve the completed result with:

result = await fabric.invocation_result(
agent_name="researcher", runtime_id=runtime_id,
request_id=request_id, artifacts="/path/to/artifacts",
)
fabric-agent-ctl result --agent researcher --runtime-id RUNTIME_ID --request-id REQUEST_ID

A null result means no completed result is available; it does not prove the request is still running or safe to repeat. A retry of the same request identifier and input returns the saved result, or reports that the original invocation is pending. Reusing that identifier with different input is rejected. Failed or interrupted reservations are not automatically replayed.

For OpenShell, pass environment=environment instead of artifacts to query the original server through the provider. The CLI requires a retained ownership record on that server. A later local SDK client can read retained files using the explicit artifact root even after runtime stop. Neither path reconstructs a missing result. Remote results retain task-side artifact paths.

Dispatched invocations that end in a host crash, host timeout, or invalid adapter response save a terminal failed result. The original call still raises its error. The saved result uses fixed diagnostic text, records unknown effects, and disables retry. A missing result never authorizes replay.