ReferenceAPIPython Library Reference

Runtime

View as Markdown

module nemo_fabric.runtime

Runtime lifecycle support for the NVIDIA NeMo Fabric Python SDK.

class RuntimeStatus

Lifecycle state of a runtime.

ACTIVE accepts invocations, STOPPED has released its runtime, and FAILED records a lifecycle failure that prevents further invocations but still permits cleanup.

Inheritance

Direct bases: str, Enum.

Values

The enum defines the following values:

NameValue
ACTIVEactive
STOPPEDstopped
FAILEDfailed

class Runtime

One logical, stateful harness execution.

Create runtimes with Fabric.start_runtime() rather than calling the constructor. A runtime serializes invocations and preserves adapter-owned harness state across turns. Use it as an asynchronous context manager to stop the runtime on exit.

Runtime-scoped overrides are recursively merged with invocation overrides; invocation values win.

property handle

Return a detached snapshot of the runtime handle.

property invocations

Return copied request, runtime, and invocation IDs for completed turns.

property messages

Return a deep copy of the latest harness-provided message history.

property runtime_id

Return the unique identifier for this started runtime lifecycle.

property status

Return the current ACTIVE, STOPPED, or FAILED state.

property supports_streaming

Return whether NVIDIA NeMo Relay ATOF streaming is enabled.

method invoke

1async def invoke(*, input: Any = None, request: RunRequest | None = None) -> RunResult

Run one turn on this runtime.

input and request are mutually exclusive. Runtime overrides are merged below invocation overrides from RunRequest. Concurrent turns on the same runtime are rejected.

Args:

  • input: JSON-compatible turn input.
  • request: Complete validated RunRequest.

Returns: The normalized RunResult for this turn.

Raises:

  • FabricConfigError: If request fields conflict or are not JSON-compatible.
  • FabricStateError: If the runtime is not active, is stopping, or is already running a turn.
  • FabricNativeUnavailableError: If the native extension is missing.
  • FabricRuntimeError: If native invocation fails before returning a normalized result.

method invoke_stream

1def invoke_stream(
2    *,
3    input: Any = None,
4    request: RunRequest | None = None,
5) -> InvokeStream

Start one turn and stream raw NeMo Relay ATOF records as they arrive.

input and request are mutually exclusive. The returned :class:InvokeStream yields raw ATOF dictionaries. Await stream.result() for the terminal normalized :class:RunResult.

Raises:

  • FabricCapabilityError: If the runtime was not started with NeMo Relay enabled and streaming=True.
  • FabricConfigError: If request fields conflict or are not JSON-compatible.
  • FabricStateError: If another turn or stream is active.

method stop

1async def stop() -> None

Destroy an idle runtime exactly once.

Repeated calls after a successful stop are no-ops. A failed runtime may still be stopped so its resources are released.

Raises:

  • FabricStateError: If the runtime is already stopping or has an invocation in flight.
  • FabricNativeUnavailableError: If the native extension is missing.
  • FabricRuntimeError: If native runtime shutdown fails.

This file was automatically generated via lazydocs.