Execution Model
Execution Model
NeMo Fabric separates configuration, planning, runtime lifecycle, and individual
invocations. Adapters that declare session_resume can also expose a portable
SessionRef for an explicitly selected durable conversation.
Most application code works with five objects:
Fabric is a lightweight, reusable SDK facade. It resolves configuration,
creates plans and runtimes, and provides a single-invocation convenience API,
but it does not represent a started execution and does not require cleanup. A
Runtime owns stateful execution and shutdown, so it is the object used as an
async context manager.
RuntimeHandle and InvocationHandle carry lifecycle identity across the
native boundary. Most Python callers use their runtime_id and
invocation_id through Runtime and RunResult rather than manipulating the
handles directly.
A runtime is a logical execution boundary, not necessarily an operating-system process. An adapter may use an in-process SDK, a process, or shared service infrastructure while preserving isolated state for each NeMo Fabric runtime.
Runtime hosting is selected by the adapter kind, not by a public FabricConfig
setting. Every local Process or Python adapter implements the persistent
local-host wire protocol. NeMo Fabric starts one host during
start_runtime(...), reuses its adapter-owned native resources across ordered
invocations, and attempts to release them during stop(). The lifecycle start
operation carries the resolved configuration and capability plan. Each
subsequent AdapterInvocation carries only runtime_context and request.
Third-party local adapters must implement the persistent local-host contract. Adapters that advertise service support can also prepare a NeMo Fabric-owned service or attach to a caller-owned service. Multiple runtimes can connect to one service while retaining distinct runtime IDs and harness sessions. Service handles are process-local in this release and must not be persisted for cold resume. A crashed persistent host is terminal for its runtime or service. The same applies when the host exceeds the protocol response timeout. NeMo Fabric does not silently respawn the host or replay a request.
Harness-native threads, sessions, and conversations remain adapter-owned state associated with the NeMo Fabric runtime. They are not additional NeMo Fabric lifecycle objects.
NeMo Fabric provides the runtime contract. Applications own scheduling, queues, retries, worker scaling, and the number of runtimes to run.
API Inventory
Create Fabric() as the primary SDK entrypoint. It is a regular Python object,
not a lifecycle context manager, and may be reused to plan, diagnose, or start
multiple independent runtimes.
The following table summarizes the SDK entry points and runtime methods: