NVIDIA NeMo Fabric Python SDK

The Python SDK is the application-facing interface for NeMo Fabric. Use it to configure an agent harness, inspect the resolved plan, run one request or a multi-turn runtime, and collect normalized results, events, artifacts, and telemetry references.

The SDK is config-first. Applications should construct a Pydantic FabricConfig from their own job, deployment, or evaluation config. Examples, CI jobs, and applications construct the typed config directly and can use ordinary Python functions to create variants.

Generated API reference pages remain the source of truth for exact signatures. This guide explains how the pieces are intended to fit together.

For installation and a package-backed example, start with the NeMo Fabric overview.

Start With One Run

This example uses an NVIDIA-hosted model. Export your NVIDIA API key before running it:

$
export NVIDIA_API_KEY=...

Construct a typed config, then pass it to Fabric.run(...). The SDK starts a runtime, invokes it once, collects the result, and stops the runtime.

1
import asyncio
2
3
from nemo_fabric import (
4
    Fabric,
5
    FabricConfig,
6
    HarnessConfig,
7
    InstructionConfig,
8
    InstructionsConfig,
9
    MetadataConfig,
10
    ModelConfig,
11
    RuntimeConfig,
12
)
13
14
config = FabricConfig(
15
    metadata=MetadataConfig(name="review-agent"),
16
    harness=HarnessConfig(adapter_id="nvidia.fabric.hermes"),
17
    instructions=InstructionsConfig(
18
        system=InstructionConfig(
19
            content="Review code for concrete correctness risks.",
20
            mode="replace",
21
        )
22
    ),
23
    runtime=RuntimeConfig(max_turns=8),
24
    models={
25
        "default": ModelConfig(
26
            provider="nvidia",
27
            model="nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
28
            api_key_env="NVIDIA_API_KEY",
29
        )
30
    },
31
)
32
33
34
async def main() -> None:
35
    fabric = Fabric()
36
    result = await fabric.run(
37
        config,
38
        input="Review the workspace changes.",
39
    )
40
41
    print(result.status)
42
    print(result.output.response)
43
44
45
asyncio.run(main())

Use base_dir to resolve relative paths in an in-memory config. Before running in a new environment, call plan(...) to inspect adapter selection and doctor(...) to check runtime requirements.

The remaining examples reuse this config value.

Execution Model

NeMo Fabric separates configuration, planning, runtime lifecycle, and individual invocations. It does not expose a separate portable session layer.

Most application code works with four objects:

Fabric is a lightweight, reusable SDK facade. It resolves configuration, creates plans and runtimes, and provides a single-invocation convenience API, but it does not represent a started execution and does not require cleanup. A Runtime owns stateful execution and shutdown, so it is the object used as an async context manager.

RuntimeHandle and InvocationHandle carry lifecycle identity across the native boundary. Most Python callers use their runtime_id and invocation_id through Runtime and RunResult rather than manipulating the handles directly.

A runtime is a logical execution boundary, not necessarily an operating-system process. An adapter may use an in-process SDK, a process, or shared service infrastructure while preserving isolated state for each NeMo Fabric runtime.

Runtime hosting is selected by the adapter kind, not by a public FabricConfig setting. Every local Process or Python adapter implements the persistent local-host wire protocol. NeMo Fabric starts one host during start_runtime(...), reuses its adapter-owned native resources across ordered invocations, and attempts to release them during stop(). The lifecycle start operation carries the resolved configuration and capability plan. Each subsequent AdapterInvocation carries only runtime_context and request.

Third-party local adapters must implement the persistent local-host contract. NeMo Fabric does not currently define a remote-service adapter contract. A crashed persistent host is terminal for that runtime. The same applies when the host exceeds the protocol response timeout. NeMo Fabric does not silently respawn the host or replay a request.

Harness-native threads, sessions, and conversations remain adapter-owned state associated with the NeMo Fabric runtime. They are not additional NeMo Fabric lifecycle objects.

NeMo Fabric provides the runtime contract. Applications own scheduling, queues, retries, worker scaling, and the number of runtimes to run.

Configure Agents In Code

Normalized Configuration Compatibility

NeMo Fabric-owned fields such as workspace, environment variables, and the invocation deadline apply to every local adapter. Adapter-translated fields are validated against the selected adapter descriptor during planning.

Core means NeMo Fabric owns the behavior and applies it uniformly before or around adapter execution. Yes means the adapter translates the normalized field into its harness. No means an explicitly configured value fails planning instead of being ignored. The following table groups provider-specific Relay subfields and additive extension maps because their support does not vary by adapter:

Model selection is deterministic: the default role wins; otherwise a single named role is selected. More than one role without default fails planning.

If a normalized field has no complete mapping, plan(...) and runtime start fail with a configuration compatibility error naming the adapter and field. doctor(...) retains the incompatibility and returns a failed diagnostic check so callers can inspect all preflight findings. NeMo Fabric does not silently drop the field. runtime.max_turns is optional, so a config that omits it remains portable across adapters with different native turn-limit support.

Build the complete nested FabricConfig directly, or start with a base config and use helpers to add capabilities. For example, extend the config from the first example with skills, MCP, and telemetry:

1
capability_config = config.model_copy(deep=True)
2
capability_config.add_skill_path("./skills/code-review")
3
capability_config.add_mcp_server(
4
    "github",
5
    transport="streamable-http",
6
    url="${GITHUB_MCP_URL}",
7
    exposure="harness_native",
8
)
9
capability_config.enable_relay(
10
    project="fabric-review",
11
    output_dir="./artifacts/relay",
12
)

Tool names are adapter-native selectors. NeMo Fabric does not define a portable catalog that translates names between harnesses. Configure an allowlist, a blocklist, or both:

1
from nemo_fabric import ToolsConfig
2
3
def with_tool_policy(base):
4
    tool_config = base.model_copy(deep=True)
5
    tool_config.tools = ToolsConfig(
6
        enabled=["Read", "Edit", "Bash"],
7
        blocked=["WebFetch"],
8
    )
9
    return tool_config

tools.enabled=None preserves the harness default. An empty list disables every executable tool. A tool cannot appear in both lists. The selected adapter must enforce the complete configured policy or planning fails. Hermes interprets these adapter-native selectors as Hermes toolset names.

Config helpers edit the typed config before planning or starting a runtime. They do not modify already-started runtimes. Use block_tools(...), remove_mcp_server(name), and remove_skill_path(path) to edit a copied config. Telemetry is enabled by adding entries to telemetry.providers; settings specific to NeMo Relay live in the top-level relay block.

For evaluation or deployment variations, use ordinary Python functions and copies of the typed config. Supply the complete final config to NeMo Fabric.

1
def review_agent_config(base, *, github_mcp: bool, relay: bool):
2
    config = base.model_copy(deep=True)
3
    if github_mcp:
4
        config.add_mcp_server(
5
            "github",
6
            transport="streamable-http",
7
            url="${GITHUB_MCP_URL}",
8
            exposure="harness_native",
9
        )
10
    if relay:
11
        config.enable_relay(project="fabric-review", output_dir="./artifacts/relay")
12
    return config
13
14
variant = review_agent_config(config, github_mcp=True, relay=True)

NeMo Relay observability is represented directly in the SDK config’s top-level relay block. ATOF uses NeMo Relay 0.6 file and stream sinks:

1
from nemo_fabric import (
2
    RelayAtofConfig,
3
    RelayAtofFileSinkConfig,
4
    RelayAtofStreamSinkConfig,
5
    RelayObservabilityConfig,
6
)
7
8
relay_config = config.model_copy(deep=True)
9
relay_config.enable_relay(
10
    output_dir="./artifacts/relay",
11
    observability=RelayObservabilityConfig(
12
        atof=RelayAtofConfig(
13
            enabled=True,
14
            sinks=[
15
                RelayAtofFileSinkConfig(
16
                    output_directory="./artifacts/relay",
17
                    filename="events.atof.jsonl",
18
                    mode="overwrite",
19
                ),
20
                RelayAtofStreamSinkConfig(
21
                    url="http://localhost:4319/events",
22
                    transport="ndjson",
23
                ),
24
            ],
25
        ),
26
    ),
27
)

Additional NeMo Relay plugin components can be supplied generically with RelayComponentConfig when their component package is available in the runtime environment.

The repository’s code-review example uses this pattern for complete Hermes Agent, Codex, Deep Agents, environment, MCP, and telemetry variants.

If a config contains relative paths, pass a base_dir to plan(...), doctor(...), run(...), or start_runtime(...). The base directory anchors skills, workspaces, artifacts, and other relative paths to the caller’s package or job layout.

API Inventory

Create Fabric() as the primary SDK entrypoint. It is a regular Python object, not a lifecycle context manager, and may be reused to plan, diagnose, or start multiple independent runtimes.

The following table summarizes the SDK entry points and runtime methods:

Single-Invocation Runs

Use run(...) when the application has one input and does not need to preserve runtime state after the result is collected.

1
from nemo_fabric import Fabric, RunRequest
2
3
request = RunRequest(
4
    input="Review the workspace changes.",
5
    request_id="request-123",
6
    context={"source": "review-service"},
7
)
8
9
fabric = Fabric()
10
result = await fabric.run(
11
    config,
12
    base_dir="/workspace/review-agent",
13
    request=request,
14
)
15
16
print(result.status)
17
print(result.output.response)
18
print(result.artifacts)

Use input=... for the common case. Use request=RunRequest(...) for structured invocation metadata. Applications read files themselves and pass either the loaded input or a validated request to NeMo Fabric.

NeMo Fabric generates runtime and invocation IDs for lifecycle correlation. An application may include its own identifiers in opaque request metadata, but NeMo Fabric does not interpret them as job, session, scheduling, or resume state.

Multi-Turn Runtimes

Use start_runtime(...) when the selected harness should keep state across turns. Every call creates a new logical NeMo Fabric runtime; callers reuse the returned object rather than selecting it with a job or session ID. The runtime stops when its async context exits.

1
from nemo_fabric import Fabric
2
3
fabric = Fabric()
4
async with await fabric.start_runtime(
5
    config,
6
    base_dir="/workspace/review-agent",
7
) as runtime:
8
    first = await runtime.invoke(input="Inspect the repository")
9
    second = await runtime.invoke(input="Now review the latest patch")
10
11
print(first.status, second.status)

The adapter reuses its native state between calls. Codex maps the calls to turns on one live thread, Deep Agents invokes one compiled graph and checkpointer, Hermes Agent reuses one agent and session database, and Claude keeps one connected SDK client. Harness-native identifiers remain adapter-internal.

NeMo Relay Streaming

Enable NeMo Relay before starting a runtime to consume raw ATOF records while a turn runs. Runtime.supports_streaming reports whether this NeMo Relay path is available. It is separate from RuntimeCapabilities.streaming, which describes adapter-native progressive output. This separation is intentional: Runtime.invoke_stream() exposes only ATOF records generated by NeMo Relay. A future normalized NeMo Fabric streaming contract will address adapter-native progressive output, such as Codex app-server message and item deltas.

The following example streams one invocation and collects its terminal result:

1
import asyncio
2
3
from nemo_fabric import (
4
    EnvironmentConfig,
5
    Fabric,
6
    FabricConfig,
7
    HarnessConfig,
8
    MetadataConfig,
9
    ModelConfig,
10
)
11
12
13
async def main() -> None:
14
    config = FabricConfig(
15
        metadata=MetadataConfig(name="streaming-agent"),
16
        harness=HarnessConfig(adapter_id="nvidia.fabric.codex"),
17
        models={
18
            "default": ModelConfig(provider="openai", model="openai/gpt-5.4")
19
        },
20
        environment=EnvironmentConfig(provider="local", workspace="."),
21
    ).enable_relay()
22
23
    async with await Fabric().start_runtime(config, streaming=True) as runtime:
24
        stream = runtime.invoke_stream(input="Review the latest patch")
25
        async for record in stream:
26
            print(record)
27
        result = await stream.result()
28
        print(result.status, result.output)
29
30
31
asyncio.run(main())

invoke_stream(...) is synchronous and starts the invocation in the background. The returned InvokeStream is an async iterator of raw NeMo Relay ATOF dictionaries. The terminal RunResult stays out of band and is available only through await stream.result(). Treat that terminal result as authoritative: replace any provisional rendering instead of appending it. Calling stream.result() does not finalize unread stream records. Calling Runtime.stop() after the result completes finalizes the stream and discards those unread records.

Streaming has the following v0.1 constraints:

• Streaming requires two explicit settings: enable NeMo Relay in FabricConfig and pass streaming=True to start_runtime(...). The streaming=True flag does not enable NeMo Relay by itself; without NeMo Relay telemetry, start_runtime(...) raises FabricConfigError. With both settings, the SDK binds its HTTP listener and injects the reserved ATOF stream sink.
• The default streaming=False leaves the existing NeMo Relay configuration unchanged; it neither enables nor disables ATOF. It also avoids the additional listener HTTP and JSON parsing cost for runtimes that use only ATIF, OpenTelemetry, OpenInference, or application-configured ATOF sinks.
• One invocation can be active on a runtime. Fully consume the stream or call await stream.aclose() before starting another turn.
• Breaking an async for loop does not finalize the stream. Call await stream.aclose() explicitly. It waits for the invocation to finish and discards unread records; it does not cancel the harness turn.
• The end of async iteration only means that no more ATOF records are available. It does not indicate invocation success. Always await stream.result(). Invocation exceptions raise from that call, while harness-reported failures are represented by the returned result’s status and error fields.
• The SDK limits its queue to 1,024 records and 16 MiB of encoded record data. It rejects individual records larger than 1 MiB and applies TCP backpressure. NeMo Relay can drop records if a consumer stalls longer than its delivery timeout.
• ATOF granularity depends on the NeMo Relay integration. Gateway harnesses expose per-delta event structure, but current ATOF records keep token text in the terminal scope. In-process harnesses expose scope-level progress.
• Reconstruct nested and parallel work with uuid and parent_uuid. Stream order alone does not define the Deep Agents scope tree.
• The listener correlates records to one NeMo Relay scope tree. Deep Agents and Hermes roots carry the NeMo Fabric request ID. For Claude and Codex, the SDK matches NeMo Relay turn scopes by their role and 1-based turn index. Records outside the matched root and its descendants are discarded, so delayed records from another turn do not enter the active stream. If the gateway turn sequence does not align with the NeMo Fabric invocation sequence, the SDK yields no uncorrelated records and emits a RuntimeWarning after natural stream exhaustion. A short drain window only collects late records from the matched tree; it does not define turn ownership.
• The SDK listener binds to the address in NEMO_FABRIC_STREAMING_HOST, which defaults to 127.0.0.1. Override it when a Claude or Codex gateway must reach the SDK through another network interface. Because the listener accepts ATOF HTTP posts, restrict access to the configured interface. If async iteration reaches the post-turn drain timeout without a NeMo Relay connection, or if NeMo Relay sends data but no record matches the active turn, the SDK emits one RuntimeWarning for that failure mode. It also warns if a NeMo Relay upload terminates before completing its chunked request body because the yielded stream can be incomplete. A caller that only awaits stream.result() still receives the terminal result and does not run this warning check.

Claude and Codex streaming use the NeMo Relay nemo-relay gateway CLI and require a stream-sink-capable release from 0.6.0 up to, but not including, 0.7.0. Follow the NeMo Relay CLI installation instructions to provision it. Hermes Agent and Deep Agents use their in-process NeMo Relay integrations.

Application-Owned Parallelism

Applications create independent runtimes when they want parallel work. NeMo Fabric does not own a queue, worker pool, semaphore, retry policy, timeout policy, or numeric concurrency limit. Each Runtime accepts one invocation at a time so its ordered harness state cannot be changed by two calls at once. If an application overlaps calls on the same Runtime, the second call raises FabricStateError. To perform work in parallel, start independent runtimes; the application decides how many to run.

Async lifecycle calls run blocking native work outside the Python event loop, so independent runtimes can make progress concurrently. This does not add a NeMo Fabric concurrency limit or scheduler.

1
import asyncio
2
3
4
async def review_one(prompt: str):
5
    return await Fabric().run(config, input=prompt)
6
7
8
async def main() -> None:
9
    results = await asyncio.gather(
10
        review_one("Review patch A"),
11
        review_one("Review patch B"),
12
    )
13
    print([result.status for result in results])
14
15
16
asyncio.run(main())

For example, each Harbor job starts an independent NeMo Fabric runtime. Harbor owns job IDs and concurrency policy; NeMo Fabric does not use a job ID to select or resume runtime state.

Unified Run Results

Every invocation that reaches the adapter boundary returns a normalized RunResult, even when the harness invocation itself failed. Inspect status, error, events, and artifacts first, then process output when the status is successful.

Important fields:

These are structured correlation fields, not interchangeable metadata: runtime_id identifies the runtime lifecycle, invocation_id identifies one invocation within that runtime, and request_id correlates the caller’s request. NeMo Fabric-generated values use type-specific prefixes such as runtime-, invocation-, and request-; callers may provide their own request_id. Consumers should store and log each field separately and otherwise treat its value as opaque rather than parsing the identifier encoding.

If NeMo Fabric cannot resolve config, start a runtime, or obtain a normalized result, the SDK raises a FabricError subclass instead of returning a partial RunResult.

Events

Each RunResult includes the normalized events collected for that invocation. Event kinds and detail may vary by adapter, but their lifecycle and correlation fields use the common NeMo Fabric contract.

Events are useful for:

• rendering invocation history in application or service UIs;
• forwarding logs and status to evaluation harnesses;
• correlating runtime, invocation, adapter, and telemetry IDs;
• reporting structured failures alongside the terminal result.

Feature Support Across Harness Adapters

The SDK presents one consistent shape across adapters, but adapters differ in their runtime requirements, accepted configuration, and optional capabilities.

Use plan(...) and doctor(...) before relying on optional features:

1
fabric = Fabric()
2
plan = fabric.plan(config)
3
report = await fabric.doctor(config)
4
5
print(plan.adapter.adapter_id)
6
print(report.status)

Use the plan to confirm adapter selection and capability routing. Use the doctor report to catch unresolved adapter descriptors, unsupported normalized capabilities, missing declared requirements, and environment problems before starting a runtime.

Capability routes assign execution ownership; they do not describe network routing. harness_native means the selected adapter executes the capability through its harness. fabric_managed means NeMo Fabric executes it outside the harness-native surface. unsupported means neither side can execute it. Routes apply to tools, skills, and MCP servers. Scalar fields such as instructions.system and runtime.max_turns are validated separately against the adapter descriptor.

Install And Runtime Responsibilities

In production, the consumer or execution environment is responsible for installing NeMo Fabric, the selected harness, adapter dependencies, model access, credentials, and any required native tools. NeMo Fabric validates and diagnoses the runtime assumptions, but it does not silently install harnesses or credentials at invocation time.

Development environments may use extras, virtual environments, or local source checkouts to make iteration easy. Production environments should prefer explicit images, preinstalled dependencies, or managed deployment packages.

Runtime compatibility checks should validate:

• NeMo Fabric SDK and native extension versions;
• selected adapter version;
• selected harness version or version range;
• required environment variables or secret references;
• optional capability support such as NeMo Relay, MCP, or tool exposure.

Custom Fields And Adapter Settings

Use normalized NeMo Fabric fields for portable behavior: models, system instructions, turn limit, runtime, environment, skills, MCP, telemetry, tools, and artifacts.

Supply request context through RunRequest.context for each invocation. Request context is not part of FabricConfig.

Use harness.settings only for stable, harness-native behavior declared by the selected descriptor. Each bundled adapter declares a closed settings schema. Executable paths, Relay command discovery, state directories, and similar launch mechanics are runtime implementation details rather than public adapter settings.

Use FabricConfig.metadata for human-readable agent identity and caller-owned annotations. NeMo Fabric preserves these values in the resolved configuration but does not copy them into RunResult.metadata. Use request_id and RunRequest.context for per-invocation correlation. RunResult.metadata contains adapter-specific result details.

Adapter settings are not portable by default. The exact descriptor selected during planning owns their JSON Schema. Planning and doctor reject settings that do not satisfy that schema. A descriptor without settings_schema accepts an empty settings map and rejects a non-empty map. Validation does not apply schema defaults or otherwise change the settings preserved in RunPlan. Validation errors identify the adapter, descriptor source and path, and offending settings path.

Errors

All public SDK errors inherit from FabricError.

Consumers own job-level retries and rollout-level failure policy. Single-invocation runs attempt to stop the runtime before returning. A Runtime used with async with also attempts cleanup after an invocation error; if cleanup fails, that failure is attached to the original exception rather than replacing it. NeMo Fabric records structured error metadata when possible and returns enough detail for the consumer to decide what to do next.

SDK Contract Boundaries

NeMo Fabric keeps a narrow execution contract. Applications own product behavior around that contract.

Schemas And Python Models

The SDK’s Pydantic models are maintained against the Rust-generated public schemas. The schemas describe the typed cross-language contracts. Use the generated API reference for exact Python signatures.

Versioning

NeMo Fabric uses explicit contract versions where persisted or independently maintained artifacts cross package boundaries:

• schema_version identifies the typed NeMo Fabric config contract transported across SDK and language boundaries.
• contract_version identifies the adapter descriptor contract implemented by a fabric-adapter.json file.
• Python and Rust package versions identify the installed SDK/core release.

NeMo Fabric versions top-level typed contracts and independently maintained adapter metadata. It does not version each config subsection independently. For example, MCP, skills, models, telemetry, and runtime fields evolve under the enclosing schema_version.

NeMo Fabric validates adapter descriptor contract versions during planning. Package semver identifies the installed implementation, but it is not the compatibility contract for independently maintained adapters.

Config Extensibility

The public schema has typed fields for stable NeMo Fabric concepts and controlled extension points for adapter- or application-owned data. Use known fields for portable concepts such as harness selection, models, runtime, skills, MCP, telemetry, and artifacts. Use adapter-owned harness.settings for settings validated by the selected descriptor and interpreted by its adapter. Use metadata or preserved extension fields for data NeMo Fabric should carry but not interpret.

Additive optional fields may be introduced within the existing document schema version when old configs remain valid. Required fields, removed fields, or semantic changes that alter how existing configs are interpreted require a new enclosing document schema version or an explicit compatibility path.

Unknown data is not the same as supported behavior. An adapter must advertise and implement a capability before NeMo Fabric treats it as runnable.

Resilience

NeMo Fabric reports lifecycle failures; applications own recovery policy. If a runtime process dies, a connection is permanently lost, or an adapter cannot complete an invocation, NeMo Fabric marks the relevant runtime/invocation as failed and returns structured error metadata when possible.

Transient I/O failures may be marked with retryable error metadata, but NeMo Fabric does not perform job-level retries by default. Consumers decide whether to retry the request, start a replacement runtime, fail the job, or escalate to a user.

Capacity And Backpressure

If a harness reports capacity pressure, an adapter should surface it as a structured error or event such as busy, rate limited, capacity exceeded, or backpressure. The consumer decides whether to wait, retry, scale out, or fail.

Next Steps

• Review the examples in the GitHub repo examples/.